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PREFACE 



The research effort represented by this report was funded 
by the Commandant of the Marine Corps. 

The objective of the research effort was to evaluate 
feedback mechanisms for MCCRES . Hopefully, the reader will 
judge that the objective has been satisfied. In our opinion 
it has been exceeded due largely to the formal and informal 
support offered us by the Marine Corps officers. 



EXECUTIVE SUMMARY 



I . Purpose of the Study 

The study was conducted at the direction of the 
Readiness Branch Headquarters Marine Corps (Code POR) . 

The study was conducted by Major T. P. Finnerty, USMC , 
Professor K. J. Euske and LtCol J. F. Mullane, Jr., USMC 
of the Administrative Sciences Department at the Naval 
Postgraduate School, Monterey, California. The study was 
used to answer the following research questions: 

(1) After MCCRES , what information should be fed back 
to the evaluated Unit Commander? 

(2) In what form should the evaluation format be? 

(3) What channel (s) should be used to provide the 
feedback to the evaluated unit? 

II . Methodology of the Study 

The study was conducted during the period July- 
September 1983. The goal of the study was to provide 
answers to the primary research questions as well as to 
three secondary research questions which will be dis- 
cussed later. Both published documents and interviev/s 
were used to develop the answers to the research questions . 
The literature reviewed for purposes of this study in- 
cluded research literature in the areas of management 
control and evaluation and DoD documents. Interviews 
were conducted with a total of 37 Marine Corps officers. 
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These officers ranged in rank from Captain to Colonel. 
Thirty-six of the officers had significant experience 
with MCCRES . Through a detailed weighting system 
emphasis was given to the responses of those officers 
who held key billets during a MCCRES. The most heavily 
weighted were those officers who had served as Battalion/ 
Squadron Commanders, Senior Evaluators and Division/Wing 
MCCRES Officers. The results of the study provided the 
data from which a comprehensive feedback model was de- 
signed which incorporated answers to the three primary 
research questions on feedback. 

Ill . Answers to Primary Research Questions 

The three primary research questions were: 

(1) After MCCRES, what information should be fedback 
to the evaluated unit commander? 

(2) In what form should the evaluation be? 

(3) What channel (s) should be used to provide the 
feedback to the evaluated unit? 

The overall goal of the MCCRES is not only to pro- 
vide a measure of unit readiness, but to provide infor- 
mation to the organization to improve readiness by 
applying resources to areas identified as deficient by 
MCCRES. It was evident throughout the study that the 
unit commander faced many constraints: training, time, 

material, equipment, logistics, and personnel. There- 
fore the information from MCCRES should be provided in 
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such a way as to allow for better allocation of these 
scarce resources to the unit to improve readiness . 

In both theoretical and applied models of evaluation 
information and feedback play an important role in both 
performing and using the results of an evaluation. In 
general all models of evaluation have common desirable 
characteristics such as: brevity, clarity, timeliness, 

interim products and reports, and responsiveness of 
feedback. Therefore the MCCRES should also incorporate 
these common characteristics of a good evaluation system 

Based upon literature review and the results of 
the field study interviews, a model for MCCRES feedback 
was developed. The model is presented at the end of the 
summary. Each of the eight major elements of the model 
that affect the evaluated unit are explained below. 

1 . Pre-MCCRES Briefing 

A detailed orientation several weeks prior to MCCRE 
to provide a firm foundation for all key personnel. 

The orientation should provide a detailed explanation 
of appropriate volumes of MCO 3501.2. Explanation of 
the details of each MPS and procedures for the evaluator 
allows the unit to maximize the potential from the 
MCCRE feedback. 

2 . Real-Time Feedback 

Regular verbal feedback throughout the exercise by 
evaluators at each level. At any logical break in the 
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scenario detailed evaluations of the most recent events 
should be provided by the evaluators. This provides 
data while the event is fresh in everyone's mind. 

3 . Evaluator Worksheets 

The Evaluator Worksheets should be provided directly 
to the evaluated element shortly after conclusion of the 
exercise. This provides an interim evaluation report 
and allows for maximum feed-back to the specific element 
evaluated. Worksheets of the evaluators would be 
especially valuable to subunits and attachments who 
often receive only minimal feedback under current 
procedures . 

4 . Two-way Debrief 

At the end of exercise, twelve to twenty-four hours 
after the end of the formal MCCRES, a detailed debrief 
should be held for evaluators and key players from the 
unit evaluated. Opportunities should be given for two- 
way discussion to resolve any confusion or disagreement 
particularly regarding subjective areas. The individuals 
from the evaluated unit should leave with not only a clear 
understanding of the deficiencies but also with a clear 
understanding of what caused the "No" evaluations and 
suggestions for improvement. Since the commander of the 
unit evaluated is responsible for the employment of the 
feedback, he should have the flexibility to tailor the 
debrief to his leadership style . The commander may de- 
sire to have only a few selected evaluators and officers 
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present or the unit commander may desire all evaluators 
and unit officers to be present. 

5 . Computerized MCCRES Printout 

The computerized MCCRES printout with percentile 
scores and detailed amplifying remarks should be pro- 
vided to the unit in a timely manner. It does a unit 
little good to get the detailed results when there is no 
time to correct deficiencies. What is timely? This can 
vary depending on deployment schedules, but getting a 
copy of the results in the hands of the evaluated unit 
so that action can be taken is essential. It is inex - 
cusable and unprofessional to allow staffing to delay 
this critical information . Delays also put the unit 
further away from the MCCRES and final written results 
tend to become historical documents vice a map for future 
training. 

6 . Comparative Percentile Results 

The data needed to compare units within a division or 
wing is currently available with limited research within 
the division or wing files. Additionally most division 
or wing headquarters are well aware of how their internal 
unit scores compare with other like units. Usage of the 
general MCCRES data bank to provide comparative scores, 
medians, and standard deviations should be provided to 
major headquarters staff, training, and doctrine commands 
for the limited purpose of planning future resource 
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allocation and to develop training doctrine. These com- 



parative results should be limited to Section/MPS/Task 
scores and should not include any sort of comparative 
final score for the individual units tested . Because 
of the many variations in evaluation conditions that 
exist, providing any sort of overall scores that could 
be used to compare individual units tested could well be 
misleading and have a negative impact. It is further 
recommended that these comparative results be directed 
only at division/wing level or higher commands to reduce 
the chance of any type of Battalion/Squadron unit 
comparisons . 

An example of comparative percentile results that 
could be provided to division/wing level for use in 
planning future training is presented in -Figure 1. 

PAST SIX MONTHS MCCRES RESULTS 



Division/ Overall | 
Wing Marine Co a 



Section 


2.0 


Operations Performed 


93.2% 


91.1% 


MPS 


2 .A 


Actions by Marines 


97.8% 


98.4% 


Task 


2 . A . 1 


Discipline 


86.4% 


90.2% 



Figure 1. Sample of Comparative Summary Percentile 
Results 



7 . Follow-up Report 

Currently no follow-up evaluation is made to confirm 
that the deficiencies have been corrected. Some type 
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of final check is needed. This check would best be 
accomplished with little formality. Since the unit com- 
mander has the responsibility for application of MCCRES 
feedback to improve unit readiness, it is best left to 
the unit commander to do the final check which certifies 
that deficiencies have been corrected. Resource reallo- 
cation from outside the evaluated unit likewise should 
be certified at the appropriate level. The key issue 
here is that the unit must be deployed at the highest 
level of combat readiness and verification of correction 
of MCCRES deficiencies at least on an informal basis is 
essential. This follow-up should be made as simply as 
possible. The unit commander can appropriately perform 
this certification based on his own experience and judg- 
ment. A formal follow-up inspection is neither necessary 
nor desirable . 

8 . Trend Analysis 

To gain a higher level of across-the-board readiness, 
a trend analysis would be of value to all units. The 
MCCRES data base represents a resource that could be 
used to keep the Marine Corps informed of its overall 
strengths and weaknesses. Special emphasis should be 
made on getting these results to training and doctrine 
organizations throughout the Marine Corps. 

IV. Answers to Secondary Research Questions 

The study also provided answers to secondary research 
questions. The three secondary research questions were: 
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(1) 



What is the validity of comparing MCCRES results 
given under differing conditions? 

(2) Does MCCRES contribute to effective training? 

(3) Is there the available time and follow-up to 
ensure correction of deficiencies identified 
during MCCRES? 

A first response to the question of the validity of com- 
paring MCCRES results given under differing conditions 
might be to say that such comparisons can only be made 
if complete standardization is achieved. Although com- 
plete standardization for purposes of evaluation may 
seem to be desirable, this is not feasible. Since various 
Marine Corps units (east coast, west coast, overseas) 
prepare for different missions, it is desirable to simu- 
late their expected combat employment through the use of 
different scenarios, terrain, and MPS's. Additionally 
the Marine Corps has other exercises, such as the Com- 
bined Arms Exercise (CAX's) , which are used to evaluate 
under standard scenario, terrain, and evaluators, but 
they have a more limited purpose than MCCRES. Addition- 
ally, because of the broader purposes of MCCRES, complete 
standardization is probably not desirable. For its in- 
tended purpose, MCCRES provides a relatively high degree 
of objectivity when compared with some other evaluation 
systems, such as the performance evaluation system used 
for the promotion of Marine officers, and the results 
are likely to be useful, at least for generic comparisons 
and for identifying trends. 
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In response to the question of MCCRES ' s contribution 
to effective training, it is evident from the study data 
that MCCRES has the potential to, and currently does, 
improve unit training in preparing for a MCCRES. Many 
units train to MCCRES standards and go so far as to use 
actual MCCRES Sect ion/MPS/Task/Requirements to structure 
their unit training. Therefore as the Individual Train- 
ing Standards (ITS) are developed for all units, it is 
essential that the training and evaluation development 
move in the same direction. 

The third question concerns time availability to 
correct MCCRES deficiencies. A comment often brought up 
during the study was that insufficient time to correct 
operational deficiencies exists between the time a MCCRES 
was taken and the time of deployment. This is a criti- 
cal factor after a MCCRES. It does little good to simply 
identify deficiencies of a unit. A set aside period of 
time must be provided in training schedules and used for 
remedial training in areas identified as weak. The time 
needed must be judged on a case by case basis and correc- 
tive training applied accordingly. The researchers were 
often told during the study that MCCRES is just one more 
hurdle to be jumped on the road to deployment and that 
once it is past the unit seldom looks back to correct 
operational deficiencies . No doubt more time would allow 
for more training in deficient areas. However, the 
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question of follow-up is also critical. As stated above 
at least an informal re- validation of deficient areas 
is needed to ensure the effectiveness of the MCCRES . 

The study provided detailed answers to the questions 
posed. The feedback to the unit is the key element to 
making the MCCRES of value to the unit evaluated. The 
purpose of MCCRES is to provide timely and accurate 
evaluation of the force readiness. Key to improving 
readiness is identification of strengths and weaknesses, 
and providing timely and thorough feedback to the evaluate 
unit and its chain of command. The more timely and 
thorough the feedback, the higher the likelihood of im- 
proved future readiness. The MCCRES is currently viewed 
as a sound, valuable evaluation system by those who have 
been involved in it. However, as this study has demon- 
strated, the feedback to units can and should be improved. 

V. Study Recommendations 

The feedback model developed and displayed at the 
end of the summary should be incorporated as the minimum 
acceptable procedure for providing feedback from any 
MCCRES. To accomplish this, the eight steps in the 
feedback model should be followed during each evaluation. 
The following information should be routinely provided 
as feedback: 

Pre-MCCRES briefing — to key MCCRES players from 
the evaluated unit 

Real Time Feedback — distributed to all those evaluate< 
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Two-way Debrief at end of "exercise — tailored to 
needs of the evaluated unit commander to 
ensure full understanding of evaluation results 

Computerized MCCRES Printout — provided in a 

timely manner to allow new resource allocation 
prior to deployment 

Comparative Percentile Results — to help direct 
unit training and for incorporation by indi- 
vidual training and doctrine commands 

Follow-up Report — done informally by the unit 
commander to ensure closure of the feedback 
loop by the correction of identified deficiencies 

Trend Analysis — to support future training as well 
as changes in doctrine. 

Implementation of these recommendations would suit 
the stated needs of fleet units as well as serve the 
intended purpose of MCCRES of providing an evaluation 
of unit readiness to higher headquarters. 

VI . Summary 

The purpose of the study was to determine the most 
appropriate and effective manner in which to provide 
feedback to the unit commander from a MCCRES. To this 
end research was conducted into the area of existing 
systems which measure or evaluate readiness. 

The study was approached from a multi-disciplinary 
background with a detailed emphasis on the economic 
question of how better distribution could be made of 
the scarce resources of manpower, equipment, and training 
time based on the results of MCCRES. To answer the ques- 
tions a detailed field study was conducted in which 
interviews were completed with 37 Marine officers of 
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rank from Captain to Colonel who have had involvement 
with MCCRES as key billet holders. Additionally, re- 
search literature in the areas of management control 
and evaluation of much of the original documentation 
that laid the groundwork for the development of MCCRES 
was reviewed along with interviews with some of the 
original developers of the system. 

The results of the study are displayed as an eight 
step feedback model which is based on previous research 
in the fields of management control and evaluation, as 
well as the results of the field study. These results 
are communicated to the reader as a model presented at 
the end of the summay which is comprised of the follow- 
ing elements: 

(1) Pre-MCCRES Briefing 

(2) Real Time Feedback 

(3) Evaluator Worksheets 

(4) Two-way Debrief at the end of Exercise 

(5) Computerized MCCRES Printout 

(6) Comparative Percentile Results 

(7) Follow-up Report 

(8) Trend Analysis 

The incorporation of this model as standard MCCRES 
feedback procedure will significantly enhance the value 
of the results to the evaluated unit and will improve 
the understanding of the resource allocation needs at 
all levels. 
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INTRODUCTION 



I . 

The Military has long recognized the need for forces to 
be prepared for combat. The United States Marine Corps has 
historically been considered this country's force in readi- 
ness. A means to evaluate and measure readiness, called the 
Marine Corps Combat Readiness Evaluation System (MCCRES) , was 
implemented by the Service in 1977. MCCRES was designed to 
provide readiness information to all levels of command. How- 
ever, the system's ultimate value lies in the communication of 
information to the unit evaluated and to the higher level com- 
mands that can support improved allocation of resources and, 
ultimately, a higher level of readiness. This research views 
the MCCRES from an interdisciplinary perspective and specifically 
looks at the economic consideration of improving the allocation 
of scarce resources of manpower, equipment and training time. 

A. PURPOSE 

The purpose of this study is to look specifically at the 
feedback resulting from a MCCRES evaluation and to consider 
all of the MCCRES information available to the Marine Corps 
and to determine how that information could be used to improve 
combat readiness . 

B. SCOPE AND METHODOLOGY 

Initially a detailed literature search was conducted into 
the areas of management control and evaluation. Following 
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the detailed literature search, the background and develop- 
ment of MCCRES was reviewed. The MCCRES is applied through- 
out the Marine Corps to evaluate ground and air combat, as 
well as combat service support units. Most commonly the 
MCCRES is applied at the infantry battalion and aviation 
squadron level. In total over 500 MCCRES's have been conduc- 
ted since implementation in 1977. Therefore, a sizeable body 
of Marine Corps Officers still on active duty have served in 
key billets during MCCRES evaluations. To capitalize upon 
their experience, a field study was conducted which included 
interviews with the initial developers of MCCRES (both Marine 
and civilian personnel) . In addition key questions on MCCRES 
feedback were posed to those who had been involved at all 
levels of MCCRES but with special emphasis on those who had 
served as unit commanders, senior evaluators, or division or 
squadron MCCRES officers. Based on this research the answers 
to questions posed by this study were answered. 

C. ANALYSIS 

The information obtained was subjected to two levels of 
analysis. First specific responses to the field study were 
analyzed through content analysis. Next an analysis of 
general responses to the data collected was conducted iden- 
tifying MCCRES elements which were repeatedly presented by 
those interviewed. Finally, a summary of unsolicited comments 
were compiled to provide completeness to the study and are 
suggested as areas for further research. 
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D. RESULTS AND CONCLUSIONS 



The results and conclusions are provided in the form of 
a feedback model consisting of the following eight elements: 

1. Pre-MCCRES Brief 

2 . Realtime Feedback 

3. Evaluator Worksheets 

4 . Two-way Debrief 

5. Computerized MCCRES Printout 

6. Comparative Percentile Results 

7. Follow-up Report 

8. Trend Analysis 

Overall the MCCRES is viewed as a valuable and necessary 
measure of unit readiness. However, to maximize the effec- 
tiveness of MCCRES, timely distribution of results must be made 
to the unit evaluated and all others directly in the chain of 
command. The timely distribution of results will allow for 
improved resource allocation. Specifically, the distribution 
of results has a high value for improving the use of limited 
training time and the development of doctrine that supports 
the effective and efficient use of Marine Corps resources. 

Thus, regular distribution of generic results should be pro- 
vided to all training and doctrine commands throughout the 
Marine Corps to improve areas consistently identified as 
deficient. 
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II. BACKGROUND 



The Marine Corps is interested in assessing Unit Combat 
Readiness. Readiness is generally viewed as the peacetime 
level of preparedness to go into combat. The Marine Corps 
Combat Readiness Evaluation System (MCCRES) is designed to 
measure units ' level of readiness through simulated combat 
conditions as determined by specific criteria set forth in the 
form of mission performance standards, tasks and requirements, 
and as judged by experienced, expert evaluators [Ref. 1] . 

The readiness demonstrated is actually a complex set of re- 
sults based on the effective allocation of resources by the 
service and the unit commander [Ref. 1]. Management control 
encompasses all management actions taken by the service and 
the unit's commander to bring the unit to peak efficiency. 

The evaluation system employed is a detailed effort to quan- 
titatively determine the success of that resource allocation. 
This chapter explores the key concept of readiness as it has 
historically evolved. It discusses the basics of MCCRES and 
lays groundwork for the detailed research that is directed at 
answering key questions involving feedback of MCCRES data to 
the unit commander for the specific purpose of upgrading 
deficiencies . 

A. MARINE CORPS COMBAT READINESS EVALUATION SYSTEM (MCCRES) 
Historically the Marine Corps has been called upon as a 
ready force to meet contingencies on short notice world-wide. 
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The specific Marine Corps Missions are detailed in the National 
Security Act of 1947. The unique air-ground combat team organi- 
zation in which each unit ranging in size from the Marine 
Amphibious Unit (MAU) to Marine Amphibious Force (MAF) is task 
organized with Marine Corps Air, Ground, Combat Support, and 
Combat Service Support elements making the organization uniquely 
suited to respond rapidly to U.S. global needs. Recent inci- 
dents such as the 1979 Iranian Crisis and the 1982 Lebanese 
Crisis have placed current emphasis on this historic role. 

1 . Purpose 

The operational capabilities of combat units describe 
their abilities to function for their intended purpose. It 
is, therefore, necessary that some means be used to estimate 
their ability to perform their intended mission short of actual 
combat. In 1976 the Commandant of the Marine Corps (CMC) 
identified the need to develop an operational readiness 
evaluation program. In his Posture Statement to Congress 
for 1978, General Louis H. Wilson, Commandant of the Marine 
Corps, made the following statement describing a need 
for : 

...an improved readiness evaluation system to, provide 
a timely and accurate evaluation of the readiness of the 
Fleet Marine Forces, including reserve units, to accom- 
plish assigned missions. [Ref. 1: p. I-A-l] 

The Marine Corps began using the system in July, 1978. Since 
that time the system has been used to test all Marine Corps 
Infantry Units, Fixed Wing Squadrons, Rotary Wing and Obser- 
vation Squadrons, Combat Support and Combat Service Support 
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elements, both regular and reserve, -at least once every 24 

months. In his FY-84 Posture Statement to Congress, General 

Robert H. Barrow ( then Commandant of the Marine Corps) made 

the following comment about MCCRES . 

A key element in the readiness equation continues to 
be operational performance standards. Even before 
these units deploy, we evaluate their capabilities 
to accomplish their assigned missions through the 
rigorous Marine Corps Combat Readiness Evaluation 
System. The System objectively evaluates the combat 
readiness of all active and reserve FMP (Fleet Marine 
Force) Units. [Ref. 2: p. 47] 

The goal of MCCRES is to test the combat readiness of 
Marine Corps units. These- tests have been developed to evalu- 
ate the five types of units mentioned previously. There are 
difficulties in applying a standard such as MCCRES. The real 
question that needs to be answered for the organization is: 

"Can the unit do the job?" [Ref. 3: p. 1]. Since the real 
answer can only be obtained under actual combat conditions, 
the Marine Corps must substitute simulated combat for actual 
combat. Thus the question that is answered is: "How close 

is the execution to doctrine? [Ref. 3: p. 1] . Thus for MCCRES, 
the Marine Corps tests adherence to doctrine, under simulated 
combat conditions as a proxy for the real question. 

2 . Scoring/Results 

The guidance for MCCRES is defined in detail by Marine 
Corps Order (MCO) 3501.2, Volumes I-X. This provides details 
on how the MCCRES is applied to each type of unit. The 
evaluation is broken down into ten (10) categories that are 
mutually exclusive and collectively exhaustive. 
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1 
2 . 
3 . 
4 , 

5. 

6 . 

7. 

8 . 
9 . 

10 . 



Reporting to higher level of command 
Preparing for operations 

Communicating (including communications SOP) 

Performing as Marines (e.g. discipline, dispersion, 
camouflage, concealment, using weapons) 

Delivering supporting fire 

Planning of operations 

Conforming to doctrine 

Executing operations 

Providing combat service support (including medical 
support) 



Supervising required actions by individual Marines. 

Each category corresponds to a vital aspect of the unit's 
performance [Ref. 3]. 

For an Infantry Battalion the evaluation is divided 
into 19 Mission Performance Standards, 145 Tasks, and 793 Re- 
quirements. Each Battalion is not tested on all MPS, Tasks, 
and Requirements [Refs. 1, 4] . The final result is a "Combat 
Ready/Not Combat Ready" for the unit evaluated based on the 
overall judgment of the senior evaluator. Results are also 
provided as percentile scores for the Section, MPS, and Tasks. 

The format for information provided is as follows: 

Infantry Unit 



Section 

MPS 

Task 

Requirement 



2.0 Operations Performed % Score 

2. A Actions by Marines % Score 

2.A.1 Discipline % Score 

2. A. 1.1 Self Discipline % Score* 



*Note: Only a yes/no is given for a Requirement, thus 

100% = Yes and 0% = No for each Requirement Graded 
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For example, the Infantry Evaluation is subdivided 
into four Sections: Section 2. A identifies standards applica- 

ble to all evaluations. It is divided into three MPS , the 
first of which is Action by Marines. 'Actions by Marines' is 
subdivided into 13 Tasks — the first, 'Tasks, Discipline' is 
subdivided into nine Requirements. The first of these re- 
quirements is 'Self-Discipline.' Requirements are evaluated 
as: Yes/No/Not Applicable. Section, MPS, Tasks, and Require- 

ments' overall evaluations are each given a weighted percentile 
score based on the number of Yes's for Requirements under the 
respective category. 

3 . Procedure 

The MCCRES is divided for simplicity into five phases. 
Phase One requires determining the appropriate MPS to be 
evaluated based on Marine Corps Order 3501.2, Vol . I, and the 
appropriate volume that applies to the unit being evaluated — 
for instance, air or combat-service support. Phase Two con- 
sists of briefing evaluators [Refs. 5,6]. Phase Three is the 
actual evaluation under simulated combat. During this phase 
the evaluators actually make the Yes/No evaluations of the 
requirements. They are guided by detailed Key Indicators 
(KI) which assist the evaluator in making the appropriate 
choice [Ref. 4] . Phase Four consists of compiling the evalua- 
tion results and determining the percentile scores as well as 
the "Combat Ready/Not Combat Ready" determination by the 
senior evaluator. 

Officially, there is no relationship between the numeri- 
cal score for the Battalion and the Combat Ready/Not 
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Combat Ready rating. It is possible, for example, that 
one battalion with an overall numerical score of 50 
could be judged Combat Ready, while another having a 
numerical score of 75 could be judged Not Combat Ready. 

[Ref. 7: p. 40] 

Phase Five, the final phase, is the debrief of the evaluated 
unit by the evaluators as well as the detailed written report 
(computer printout) which is provided to the evaluated unit. 
Other reports are forwarded to Headquarters Marine Corps 
(CMC) — an initial message report within 10 days and the 
detailed report within 30 working days [Ref. 1]. 



B . READINESS 

At the center of any Readiness Evaluation System is under- 
standing what is meant by the key concept of "readiness." 

The inherent difficulty in describing this term makes any 
evaluation, measure, or model of readiness all the more 
complex. Thus we must explore the meaning of "readiness." 

1. Definition 



Usage of the term "readiness" has tended to change 

even at the highest levels within the Department of Defense 

(DoD) . In FY 1977, then Secretary of Defense, Donald H. 

Rumsfeld, used the following definition of readiness: 

"Readiness" is a concept that integrates the diverse 
factors that affect the ability to deploy, engage, and 
sustain effective combat forces. It starts with the 
overall ability and proficiency of U.S. fighting men... 

An equally important determinant of overall readiness 
is the availability, capability, and condition of the 
force's fighting equipment. [Ref. 8: p. 2] 

In FY 1978 the same Secretary of Defense is quoted 

as follows: 



9 



"Readiness" refers to the capability to respond adequately 
to diverse situations and to sustain that response as 
long as necessary. The "Readiness" of Defense Combat 
Forces depends on a myriad of diverse and often inter- 
related factors. [Ref. 8: p. 2] 

Under the current administration the concept has taken 

a more standardized definition. In an address to Congress on 

February 8, 1982, and January 31, 1983, Secretary of 

Defense Weinberger used the following definition: 

Readiness is the ability of forces, units, weapons 
systems, or equipment to deliver the outputs for which 
they were designed (including the ability to deploy 
and employ without unacceptable delays) . It depends on 
having the required quantities of equipment in the hands 
of the units on a day-to-day basis and on having the 
required number of adequately trained people assigned 
with the necessary mix of grades and experience level 
and to ensure that people and machines can work to- 
gether. [Ref. 9: p. 1-28J 

This definition includes training (individual and unit), 
material, equipment, logistics, and personnel all as part 
of the readiness definition. 

For the purpose of continuity and brevity, the follow- 
ing definition of readiness is used on a day-to-day basis by 
the Marine Corps and is the accepted definition from JCS, 
Publication 6, used for the DoD Unit Reporting System 
(UNITREP) . 

Readiness: ability of forces, units, weapons systems, 

or equipments to deliver outputs for which they were 
designed (includes the ability to deploy and employ 
without unacceptable delays). [Ref. 10: p. 1] 

It is further amplified as follows: 

Readiness is essentially a measure of pre- D-Day status 
(extending at most into initial combat operations) while 
sustainability is a Post-D-Day measure. Hence, we 
often speak of peace time readiness , but combat 
sustainability . [Ref. 10: p. 1] 
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For consistency this definition of readiness, as amplified, 
is used throughout this study, since it provides the cur- 
rently accepted DoD definition. In the literature the terms 
'effectiveness' and 'sustainability' are often interchanged 
to indicate Post-D-Day combat capabilities. Because of its 
broader implications the term effectiveness is used herein 
to mean Post-D-Day status. Hereafter we refer to Pre-Combat 
Readiness and Combat Effectiveness . 

2 . Indexing 

The difficulty in defining readiness may be exceeded 
only by the difficulty of measuring it. Many attempts have 
been made to index readiness, but to date none have been suc- 
cessful or achieved wide acceptance. As described by Barzily, 
Marlow, and Zacks, in their "Survey of Approaches to Readi- 
ness," the motivating factor common to all attempts is to 
measure the effects of budgetary changes on readiness. They 
reviewed a number of readiness indexes, such as the U.S. 

Navy METRI Project, MARIS Project, MAXCAP Models, and others 
and found that: 

The readiness indexes posed do not attain the desired 
objective. They are usually very insensitive to changes 
that occur at the lower echelons and, furthermore, they 
are generally improper indexes of readiness. [Ref. 11: 
p. 25] 

3 . Economic Question 

The readiness as described in both Definitions and 
Indexing sections has strong economic overtones. The approach 
in this study is to treat readiness as an economic question 
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(i.e., the allocation of scarce resources of training time, 
material, equipment, logistics, and personnel as inputs with 
the desired results of maximizing outputs) . The true output 
of a combat organization is effectiveness in combat. There 
is obvious difficulty in measuring that output. Because of 
this difficulty proxy measures, such as number of aircraft 
or number of ships, are generally used. The difficulty of 
measuring output was recently stated in a study by ' 

Rand for the Assistant Secretary of Defense/Manpower, Reserve 
Affairs, and Logistics: 

The output of an Army Maneuver/Firepower Unit is 
especially difficult because such units have no single 
output or product that can be directly related to 
their mission. [Ref. 12: p. 7] 

In a recent study by Steven L. Funk, several observa- 
tions were 'made which have application here. He states that 
although peacetime Army readiness is an honored theme, it is 
not always practical. This is a result of preparedness not 
being the actual primary peacetime mission, but just one of 
several competing missions that take resources of manpower, 
equipment, and training time. Funk cites major factors that 
detract from the unit being able to train for combat situations: 

(1) Lack of resources to accomplish requirements 
(i.e., skilled personnel, time, ammunition, 
repair parts, and facilities), 

(2) changing and competing priorities that fragment 
resources and negate planning, and 

(3) a climate that deprives unit leadership of 
decision "maneuver space" and fragments their 
attention (i.e., centralized decision making/ 
control, burdensome administration, low unit 
cohesion and ambiguous institutional values) . 

[Ref. 13: p. 6] 
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Although the previous study was made of the U.S. 

Army, many of the detractors apply equally to the Marine 
Corps. Some policies such as Unit Rotation (i.e., the policy 
of maintaining unit integrity by replacing an entire battalion/ 
squadron rather than replacing individuals) and Deployment- 
Lock-On periods (i.e., the policy of assigning resources to 
a unit well in advance of a planned deployment and then main- 
taining and replacing those resources through the training 
cycle and into deployment) have lessened the negative effects. 
However, in the Marine Corps the problem of competing priori- 
ties will always exist. The unit commander is constrained by 
many elements beyond his control [Ref. 12]. Thus the question 
of unit readiness is certainly one of economic priorities or 
competing priorities. Readiness to a large degree depends on 
the effective resource allocation both to the unit and within 
the unit. 

C. MCCRES AS THE EVALUATION SYSTEM 
1 . Research 

The evaluation system employed by the Marine Corps 
has undergone much study from the standpoint of developing an 
objective versus a subjective system of evaluation. Much of 
the statistical work for this has been done by the George 
Washington School of Engineering and Applied Science [Refs. 

3, 4]. Additionally, the questions of evaluator selection bias 
were analyzed in other research that supported this study 
[Refs . 5 , 6 ] . 
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As a result of the George Washington University 
studies, it was determined MCCRES had the following merits: 

a. Most requirements are requests for descriptive data 
and not for judgments. Judgments were previously 
made by defining the requirements and assigning 
weights . 

b. The details of the doctrine are given and interpreted 
in the requirements, thus avoiding the possibility 

of being misinterpreted or forgotten by the 
evaluators . 

c. The execution of most requirements consumes short 
time periods and thus the evaluators ' memories are 
not overburdened. 

d. Assigning a score of a YES or NO is easier than 
assigning scores on any other scale. 

e. The set of the requirements exhausts the details 
of the executions. [Ref. 3: p. 25] 

Though the George Washington University research identifies 

merits of MCCRES, it does not address the question of what 

feedback should be provided to the evaluated unit. 

2 . Feedback 

The purpose of this research was to determine the 

types of feedback that are most useful and valuable to an 

evaluated unit. Feedback is a necessary part of any control 

or evaluation system. In December, 1981, B. General A. A. 

Sordo, then Director of the Marine Corps Training Division, 

made the following comment: 

The evaluation and feedback process in training is 
one where we need to do substantial work. MCCRES 
helps us greatly. In the future, as we link the 
individual training standards of our ITS System to 
MCCRES Mission Performance Standards, we'll be able 
to do better! But above and beyond that, our 
measurements of adequacy and effectiveness are for 
the most part decentralized and highly subjective 
and impressionistic. In the long run we rely on 
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our assessment of the unit or organizational capabili- 
ties as the best gauge of adequacy in training 
preparation. [Ref. 14: p. 54] 

D. RESEARCH QUESTIONS 

As a result of the need to define specifically the feed- 
back and information needed by evaluated units as the result 
of MCCRES , this study was undertaken. The following questions 
form the basis for this study: 

1 . Primary 

a. After a MCCRES, what information should be fed 
back to the evaluated unit commander? 

b. In what form should the evaluation feedback be? 

c. What channel (s) sliould be used to provide the 
feedback information to the unit commander? 

2 . Secondary 

In addition to the specific questions above, there are 
several subsidiary questions which arise as a logical conse- 
quence of the primary questions. They involve the following 
topics: (1) validity of comparing MCCRES results given under 

differing conditions, (2) MCCRES contribution to effective 
training, and (3) time availability and followup to ensure 
correction of deficiencies identified during the MCCRES. 

To properly answer the questions posed, the authors 
took a multi-disciplinary approach. The management control 
and evaluation literature can provide a great deal of insight 
toward understanding the complexities of the information 
needs. Thus the literature of the two fields is explored in 
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detail in the next chapter. In addition much work has been 
done in the area of modeling effectiveness. This has been 
done from the perspective of applied, conceptual, statistical 
and economic models. Since these provide rich theories for 
the understanding of the measurement and indexing of effec- 
tiveness, these concepts are explored. 

E. SUMMARY 

This chapter was designed to provide a history of MCCRES , 
explain the need for such a system from the Marine Corps 
perspective, give a basic description of the system, and 
provide a detailed understanding of the inherent difficulties 
of efforts to evaluate the concept of readiness. 

Readiness is approached from the standpoint of economics — 
the application of scarce resources (inputs) of time, men, 
materials to the process of simulated combat. The unit com- 
mander controls the organization with a highly sophisticated 
management control system in accordance with the external and 
internal objectives and policies. MCCRES is a unit evaluation 
system which evaluates the management control process of that 
organization. The primary questions are then presented as 
the basis of this research. The following chapters briefly 
explore the management control systems and evaluation 
literature with specific emphasis on feedback mechanisms. 
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III. MANAGEMENT, MANAGEMENT CONTROL, EVALUATION, 

AND FEEDBACK 



The topics to be covered in this chapter are as follows: 
first the topic of management is described in a general 
context; management control and its implications for feedback 
are explored; then, a series of selected evaluation theories 
are presented in detail. Finally a series of evaluation 
models with application to the military are provided. Empha- 
sis in these models is on feedback. A summary of feedback 
as developed in the models is presented. 

A. MANAGEMENT 

Management is described as: (1) "art or act of managing; 

conduct; control; direction. (2) Judicious use of means to 
accomplish an end; skillful treatment" [Ref. 15]. Webber 
describes it in many contexts — experience, training, practice, 
theory, art, and science [Ref. 16]. The simplest definition 
and one often heard is the art of getting things done through 
people. The elements, however, that all descriptions of 
management have in common are that managers engage in two 
important activities: planning and control [Ref. 17]. It 

is important to make the distinction clearly between the terms 
management and leadership. The military has long recognized 
that the term "leader" has much broader implications than 
that of manager. The unit is more than the sum of its parts; 
and, therefore, the military commander's responsibility is 
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not only for the technical and tactical proficiency of the 
unit, but for the overall cohesive functioning of the organi- 
zation [Ref. 18] . In this context it is appropriate to say 
that management skills are a necessary subset of leadership 
and that the good leader is necessarily a good manager, how- 
ever, the converse of that statement is not necessarily true. 



B . MANAGEMENT CONTROL 

Robert Anthony placed management control in a hierarchi- 
cal framework with Strategic Planning above and Operational 
Control below Management Control. He described this classi- 
fication for any large organization where these functions are 
present. 

Strategic planning is the highest level of management 
where the long range objectives, policies, and plans of the 
organization are made. 

Strategic planning is the process of deciding on 
objectives of the organization, on changes in the 
objectives, on resources used to attain these 
objectives, and on the policies that are to govern 
the acquisition, use, and disposition of these 
resources. [Ref. 19: p. 16] 

The management control level involves middle management, 

and the timeframe is shorter, about 12 months. 

Management control is the process by which managers 
assure that resources are obtained and used effec- 
tively and efficiently in the accomplishment of the 
organization's objectives. [Ref. 19: p. 17] 

The lowest level of control is operational where the day- 
to-day functioning of the organization is accomplished. The 
implementation of plans and policies are executed at this level 
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Lebas describes management control as an enlarged feedback 
loop which affects human behavior and performance [Ref. 20]. 
Hofstede also provides a useful definition of management con- 
trol "...as a pragmatic concern for results, obtained 
through people" [Ref. 21: p. 193]. 

Given the purpose of this study is to identify feedback 
needed for improved resource allocation, it is important to 
focus on the terms "effectiveness" and "efficiency", which 
are present in many of the definitions of management control. 
In this context the terms effectiveness and efficiency are 
defined as follows: 

Effectiveness ... relates to the accomplishment of the 
cooperative purpose .. .when a specific desired end is 
attained we shall say that the action is effective. 

[Ref. 19: p. 27] 

Ef f iciencv . . . ref ers to the engineering sense of... 
the optimum relationship between given inputs and 
outputs. [Ref. 19: p. 27] 

Lebas identifies the three major schools of thought 
on management control: organizational process approach, 

information economics approach and behavioral approach. 

1. Organizational Process Approach 

The Organizational Process Approach is that taken 
by Anthony. He divides the feedback loop into three distinct 
parts, described previously as strategic planning and manage- 
ment control and operational control. This approach reduces 
the human motivational problem to what Anthony calls "goal 
congruence" or the various ways in which the manager can be 
encouraged to take actions which are in the best interest of 
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the company. Achieving effective and efficient use of re- 
sources is accomplished through procedures within the 
organization such as performance evaluation, reporting, and 
budgeting [Ref. 20]. 

2 . Information Economics Approach 

The Information Economics Approach is more abstract. 

It proposes that only models based on expected utility maxi- 
mization need to be considered. A set of subjective probabili- 
ties are assumed to be known by the decision-maker and that 
the decision-maker will always act to maximize utility. In 
this approach, good performance is described as the action 
which minimizes the end result differences between the expected 
outcomes and the actual outcomes. The emphasis, however, is 
on the decision model not the outcome. A further aspect of 
the theory is its emphasis on maximizing the expected utility. 
This leads to the conclusion that the control process is a 
prior phenomenon that suggests management spend more time 
preparing decisions than evaluating outcomes [Ref. 20]. 

The major limitation to the Information Economics 
Approach is that it is all based on the Savage-Rational Man 
Model and that an effort will be made to maximize utility in 
every case. Although an interesting theoretical model, it 
relies heavily on the calculus of maximizing utility in each 
and every case [Ref. 20]. 

3 . Behavioral Approach 

This is actually a loosely structured set of ap- 
proaches to the concept of management control taking ideas 
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from many theories. Two descriptive ideas that it entails 
from Herbert Simon are: 

Satisficing ... interrupting the action selection process 
as soon as the first acceptable action has been found. 

And 

Bounded Reality ... the notion that man is a piecewise 
rational. [Ref. 20] 

Other ideas presented by Ouchi and referred to by 
Lebas have shown that: 

. . .rigorous output control is the most effective way 
to induce good performance in managers. [Ref. 20] 

Lebas views the real value of behavioral approaches 
in their diversity and being based on observations of people 
in real situations. The limitations come from their multi- 
plicity and that creates difficulty in setting any sort of 
coherent guidelines for a manager to employ [Ref. 20]. 

4 . Integrated Approach 

The previous broad structures of management control 
system have been integrated by Ansari in what he refers to 
as an operational systems concept [Ref. 22]. He views the 
previous perspectives on management control systems in two 
versus three broad categories. Grouping the structural view 
(Operational Process and Information Economics) and the 
behavioral views : 

a. Two Perspectives 

Ansari describes the structural view of management 
control as that adopted by researchers in cybernetics, account- 
ing, and management information systems — those that concentrate 
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primarily on the information and communications aspects of 
control systems. The behavioral approaches are identified 
as those that are based primarily on human behavior in organi- 
zations and regard control as a problem of encouraging sub- 
ordinates to achieve performance goals. In this context he 

j 

uses control systems to describe: 

...those arrangements and actions designated to 
facilitate its members to achieve higher performance 
with least unintended consequences. [Ref. 22: p. 102] 

b. Elements of Management Control Systems 

There is agreement among most authors reviewed 
from both the structural and behavioral schools that all 

1 1 

management control systems consist of two elements. The 
first being "an information network which prescribes the 
rules for measurement, collection, processing, and transmission 
of information" [Ref. 22: p. 102]. This element causes the 
information on performance, goals, outputs, and exceptions 
from plans to be transmitted to managers. The second ele- 
ment is the set of social relationships through which the 
control system achieves the organizational goals [Ref. 22]. 

c. Joint Consideration 

Ansari argues that management control is actually 
best approached from a combination of the structural and be- 
havioral ideas . He declares that the current phase of design 
is more situational and that it focuses on the more important 
issue of improving performance instead of the narrow con- 
cept of constraining behavior. The information that is pro- 
vided passes through two phases. First, it must be perceived 
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before it can be used and second not only managers, but 
subordinates, must also use information for self-evaluation 
and guidance of performance. He further identifies five con- 
siderations which are characteristic of all information 
structures . 

1. Nature of Measures — the subordinates' motivation 
is affected by the completeness, objectivity, 
and influence of the characteristics of perform- 
ance measures. 

2 . Source and Order of Presentation — The source of 
the information must be credible and the order 
of presentation can change the perceptions of 
the information. 

3. Timing — Eoth the speed and frequency are impor- 
tant, too long an interval may cause the user 
to lose interest or be distracted. 

4. Route - -The route may alter the information and 
thus the sender and receiver may not be sharing 
the same information. 

5. Shared Information — The others with whom the 
information is shared thus may affect the 
recipient's view of the information with regard 
to fairness and accuracy. [Ref. 22] 

All of these considerations must be made by the 
designer in any information, evaluation system. To summarize 
the characteristics of the integrated model, this takes into 
account both the structure of the organization and its infor- 
mational systems as well as the social side considering 
subordinate personality and leadership style. The designer 
of a management control system should combine components 
(such as an evaluation system) in such a way as to minimize 
cognitive conflicts and encourage behavior which resolves 
conflict with positive results for the organization. It 
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should also be kept in mind that if rewards for performance 
are too highly contingent on outputs of the information struc- 
ture, the greater the chance that there will be controversy 
over the output [Ref. 22] . A good example of this contro- 
versy is discussed by Anthony Hopwood in the way in which 
accounting data was used in the performance evaluation of 
managers . 

Based upon a study he conducted, Hopwood identi- 
fied four ways in which accounting data was used in evaluation. 

(1) Budget Constrained Style--required to meet a budget 
but not concerned about costs. 

(2) Budget Profit Style — concerned with both meeting a 
budget and with costs. 

(3) Profit Conscious Style — concerned with costs but not 
with meeting a budget. 

(4) Non-Accounting Style — not concerned with meeting a 
budget or with costs. [Ref. 23] 

He concluded that based on management style, many dysfunc- 
tional behaviors may occur as the result of emphasis on 
accounting data results in evaluations [Ref. 23]. 

5 . Other Thoughts on Management Control 

Management control contains many rich conepts which 
play a great part in the theory of evaluations. San Miguel 
has made several interesting observations on management 
control : 

The end is a system that enables managers to make sound 
decisions as to the efficient and effective allocation 
of human, physical, and financial resources to attain 
the objectives of the organization. [Ref. 24: p. 177] 
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San Miguel further characterizes management science's 
attempts to deal with modeling of complex control systems. 

He characterizes the quantitative and economic decision tools' 
attempt to simplify the decision making in large scale organi- 
zations as falling short of their objectives — primarily be- 
cause of the lack of knowledge of human behavior and the 
ability to quantify it. He further cites measurement and 
communications to be at the center of control systems, internal 
planning, and reporting. There is general agreement in the 
management control literature that measurement and communica- 
tions systems have an important impact on the behavior of 
individuals in organizations, their motivations and, thus, 
their performance. Therefore, the areas of measurement and 
evaluation are legitimate concerns for the design of any 
management control system [Ref. 24]. 

6 . Summary 

The intent in this section was to set the stage 
through the discussion of the broader topics, management and 
mangement control, for what is to be developed in the next 
section on evaluation. Management control is a complex topic 
which encompasses the broad aspects of all the control sys- 
tems imposed on the organization through strategic plans and 
policies as well as those developed within to assist managers 
in internal control. 

C. EVALUATION OVERVIEW 

Evaluation models directed at readiness all embody a 
common goal. Each in its own particular way attempts to 



25 



identify a definitive measure or some particular relative 
measure of the unit's Pre-D-Day preparedness for combat. 
Preparedness may be viewed as readiness which acts as a proxy 
measure for the military unit's ability to perform in combat. 
Similar efforts are made to evaluate the ability to perform 
in civilian organizations. In its simplest form, output is 
measured by production level or quantity produced by a produc- 
tion facility in a given time period. Various attempts are 
also made to measure profitability of the firm. Microeconomic 
theory assumes that each firm is attempting to maximize profit 
[Ref. 25]. Therefore, some measure(s) are applied to the 
fiscal results of each period to determine the financial 
performance of the organization during the period [Ref. 25]. 
Similarly it can be assumed that each unit tries to maximize 
its readiness. This section reviews efforts that have been 
made to evaluate, measure, and model performance both in 
theory and practice. 

1 . Evaluation Defined 

The term evaluation is defined differently by various 
authors; however, Stuff lebeam et al. brings together three 
generally accepted definitions of evaluation and provides a 
comparison . 

a. Measurement Definition 

Evaluation is identical to measurement. It 
builds on attempts to measure psychological attributes or 
characteristics [Ref. 26: pp . 9-10]. 
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b. Congruence Definition 

Evaluation is a comparison of the congruence 
between performance and objectives. This provides emphasis 
on objective performance also [Ref. 26] . The measurement 
definition considered evaluation to occur after the fact and 
to measure some attribute. In contrast, the congruence 
definition refers to on-going evaluation in meeting the 
objectives. According to Stufflebeam, organizations pick 
objectives which are specific and have objective measurable 
results. To make the congruence definition work, there must 
be some selection of objectives to be measured. Stufflebeam 
further indicates that the congruence definition creates a 
need for the evaluator to find short term measures of per- 
formance. These indicators, identifiable outputs, or behaviors 
come to be viewed as performance. These identifiable outputs 
become the ultimate criteria of all organizational decisions. 

The congruence definition has advantages, but the need to 
find measurable objectives creates problems [Ref. 26]. 

c. Judgment Definition 

The professional judgment definition allows full 
evaluation of all organizational attributes, both quantifiable 
and non-quantif iable , and is easy to implement [Ref. 26]. 

This definition is often applied where the dimensions to be 
measured are difficult to define and more difficult to quan- 
tify. The judgment definition has its problems. The evalua- 
tion is, by definition, based upon the judgment of an individual 
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which lacks objectivity identifying what data are used and 
how they are used [Ref. 26] . 

Table 3.1, taken from Stufflebeam, provides a 
comparison of advantages and disadvantages of each definition. 
2 . Principles of Evaluation 

For an evaluation to be effective it must be consis- 



tent with the purposes, objectives, and goals of the activity 
being evaluated [Refs. 5,27,28]: 



a. Evaluations must be conducted in terms of purpose-- 
the evaluator and the evaluated must fully participate 
and work for the common goal. 

b. Evaluation must be cooperative — all involved as both 
the evaluators and evaluated must fully participate 
and work for the common goal. 

c. Evaluation must be continuous — it must be on-going; 
a one time effort with no followup is an affront to 
the professional concepts. 

d. Evaluation must be specific — specificity is the key, 
generalizations do little good to help remedy defi- 
ciencies or to identify true strengths. 

e. Evaluation must provide the means and focus for 
trainers to be able to appraise themselves, their 
practices, and their products. 

f. Evaluation must be based upon uniform and objective 
methods and standards . 



3 . Summary 

Definitions and principles of evaluation were pre- 
sented in this section. A single commonly agreed upon defini- 
tion of evaluation does not exist. There is more general 
agreement on the principles of evaluation and to a large 
degree these embody a common sense approach for the usefulness 
of any evaluation. Next, four evaluation theories are presenter 
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TABLE 3.1 



Advantages and Disadvantages Accruing from Different 
Definitions of Evaluation 



Advantages 

Builds directly on scientific 
measurement 

Objective 

Reliable 

Data mathematically 
mampulatable 

Norms and standards 
emerge 

High degree of integration with 
the process 

Data available on process and 
structure 

Possibility of feedback 



Easy to implement 

Brings all variables 
into consideration 

Takes experience and expertise 
into account 

No time lag while waiting for data 
analysis 



Disadvantages 

Narrow instrumental focus 

Inflexible because of time 
and cost to produce new 
instruments 

Obscures judgments and the 
criteria for maxing them 

Eliminates variables currently 
considered as not measurable 
or labels them unimportant 

Places evaluator in tecnmcal 
role 

Focuses narrowly on objectives 

Elevates behavior as the ultimate 
criterion of every action 

Focuses on evaluation as a 
terminal process 

Dictated mainly because of 
ignorance or lack of sopnistication 

Questionable reliability 

Questionable objectivity 

Not susceptible to ordinary 
scientific, prudential measures 

Both data and criteria are 
ambiguous 



Generalization very difficult 



D. EVALUATION MODELS 



There are diverse models and applications of evaluation. 
This discussion is not intended to be all inclusive. What 
is intended is to introduce a variety of models which exist 
and to show their relationship to the questions pursued in 
this research. 

1 . Management by Objective (MBO) 

In recent years probably no system of personnel 
management has gained more attention than Management by 
Objective introduced by Peter Drucker [Ref. 29]. The MBO 
approach is applied to individual as well as organizational 
evaluation systems. 

The MBO Approach assumes that one would perform more 
effectively because you have planned your own objec- 
tives and could control your own behavior. [Ref. 16: 
p. 316] 

MBO takes a humanistic or human values approach to 
managing people. The basic steps in MBO are as follows: 

1. The subordinate proposes goals for the next time 
period. 

2. The subordinate and superior discuss, modify, and 
reach agreement on the goals. 

3. Periodic formal and frequent informal review of the 
goals and progress toward the goals. 

4. Subordinate reports on performance at the end of 
the period. 

5. Repeat the cycle. [Ref. 16: p. 316] 

MBO deals well with a number of problem areas in evaluation 
systems. It is designed to facilitate two-way communication. 
MBO has found success in both the private and public sector. 
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In their article on "Employee Growth Through Performance 
Management," Beer and Ruhr found success with employment of 
MBO at Corning Glass Co. [Ref. 30]. In addition, the Depart- 
ment of Health, Education, and Welfare found it a most valua- 
ble system in pursuing management and control of their many 
diversified programs [Ref. 31]. Figure 3.1 provides a simple 
model of how MBO is designed. It should be noted that the 
MBO model focuses on performance as identified and measured 
through objectives agreed upon by supervisor and subordinate 
[Ref. 32]. 

Feedback in the MBO System is scheduled at regular 
intervals. The meetings can be used to compare results to 
the objectives set as well as to help make future objectives 
more realistic. The objectives themselves are determined by 
higher level management in coordination with the employee 
or unit. To a great extent the comparison of goals set and 
achieved are reported by the individual being evaluated. 
Typically the manager submits an evaluation of each objective 
based on the following classification system: 

1. Satisfactory 

2. Minor Problem 

3. Major Problem 

Feedback is handled by face to face conferences between the 
evaluator and employee. 

In a recent Marine Corps Gazette article. Major James 
M. Clarke reflected his good experiences with the MBO merit 
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Source •• 



Adapted from 




Figure 3.1. Sample MBO Model 
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pay system used with high level GS employees. He referred 

to MBO in the context of a personnel evaluation system; 

however, it also has application here: 

To improve performance counseling, we must get away 
from subjective judgments through a more formal system 
which concentrates on objectives. The merit pay 
system provides a good model. [Ref. 33: p. 47] 

Major Clarke's article provided support for the MBO 
method because clear objectives are established and regular 
communications are encouraged. In the Marine Corps' personnel 
system, as in other areas of inspection and evaluation, pro- 
fessional judgment has historically been used as the primary 
method of evaluation and lack of specific objectives has 
reinforced the reliance on the judgmental method. 

MBO places a great emphasis on the objectives and the 
regular feedback on how the evaluated worker is measuring up 
to those objectives [Ref. 16]. MBO further emphasizes dis- 
playing the individual's strengths and weaknesses to himself, 
rather than to others. Although MBO emphasizes timely, 
accurate, and objective feedback, the details of that feed- 
back can only be spelled out when a system is input to a 
specific organization [Ref. 30], It further encourages the 
feedback be given on a continual basis, not "saved up for an 
end of the year inquisition" [Ref. 16: p. 317]. 

2 . Evaluation and Authority 

A comprehensive and ongoing study of evaluation and 
authority was conducted by Scott and Dornbusch (1977). In 
their study of professional organizations, results were 
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gathered from university faculty members, clergy, public 

school teachers, principals, nurses, and others. They found: 

Organizations are power structures in which some partici- 
pants give differential access to organizational 
rewards and penalties in order to control other 
participants. [Ref. 34: p. 134] 

The Scott-Dornbusch studies present a conceptual model 
of the relationship of evaluation and authority. As a result o 
these studies, they developed the model presented in Figure 
3.2. The model is a systems analysis approach which provides 
for monitoring and regulating by means of a feedback loop. 

This approach concentrates on the involvement of each actor 
in the process. The actors in the Scott-Dornbusch model are 
described as follows: The allocator gives the responsibility 

for performing the task. The criteria setter is concerned 
with assessing the effectiveness and efficiency of a task 
performance. To a large measure the criteria is determined 
by the goal toward which the performance ■ is directed. The 
determination of which information to use in order to arrive 
at a performance evaluation is the responsibility of the 
sampler . The appraiser uses the information on the sampled 
indicators and transforms the observed values into scores. 

The goal of the evaluation process is to provide a measure 
of the performer based on performance and outcome [Ref. 34]. 

The manager must get each of these individuals in the decision- 
making process and coordinate their activities. Emphasis is 
placed in this model on communications among the right-holders. 
The right-holders are those given authorized power or authority 
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Noture of 




Source •• Adapted from [Ref. 34 : p. 20 j 



Figure 3.2 Model of Evaluation Process with Communication 
Links Among Right- Holders. 
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rights by the organization. The authority rights are granted 
to the allocator, criteria setter, sampler and appraiser. 
There are a variety of interconnecting links which may exist 
between these r ights-holders which define the authority 
structure within any given organization. Thus the formal 
evaluation process is based on the structure of the inter- 
connection among the right-holders [Ref. 34]. 

The Evaluation and Authority Feedback Model describes 
performance by direct and indirect measures. An example of 
a direct measure would be the number of units produced as a 
measure of labor productivity. An indirect measure is the 
number of professional papers written to evaluate the produc- 
tivity of a scientist or scholar [Ref. 34]. The variety of 
samples requires judgment on the part of the evaluator to 
determine what is high or low performance so judgment must 
be used by the evaluator. Thus, accurate appraisal of the 
sample taken requires a complete knowledge of the task, the 
performance, and the specific circumstances. Based on all of 
these measures it is decided what, if anything, should be 
conveyed to the performer concerning quality of the task 
performed [Ref. 34]. 

3 . Practice of Program Evaluation 

Anderson and Ball take an applied approach to program 
evaluation. They identify six specific purposes of program 
evaluation [Ref. 35]: 

1. to contribute to decision about program installation, 

2. to contribute to decisions about program continuation, 
expansion, or certification. 
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3. to contribute to decisions about program modifications, 

4. to obtain evidence to rally support for a program, 

5. to obtain evidence to rally opposition to a program, 

6. to contribute to the understanding of basic psychologi- 
cal, social, and other programs. [Ref. 35: p. 4] 

Anderson and Ball emphasize the practice aspect that 
evaluations can address a wide variety of questions and pro- 
vide many useful services. Their experience is primarily 
with social programs, but it is applicable to this study. 

They identify a total of seven types of evaluation methods: 

1. The experimental and quasi-experimental study, 

2. Correlation methods, 

3. Surveys, 

4. Personnel or client assessment, 

5. Systematic expert judgment, 

6. Clinical or case studies, 

7. Informal observation and testimony. 

The real richness of the ideas presented by Anderson 
and Ball involve their emphasis on feedback which they refer 
to as the: Communication and Dissemination of Results. They 

emphasize the need for communications of the results to be 
bi-directional. The effective communications of results are 
viewed as a sign of the positive health of the program. Evalu- 
ators should take pride in presenting and disseminating their 
results. The dissemination is more than merely a phase tacked 
on the end of a program; it involves more than simply telling 
the findings. At a minimum, what the dissemination involves 
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is that the results should include information about the 
evaluation plans and procedures as well as the findings. 

There are also several criteria provided to guide the dis- 
semination to various audiences. First, if an evaluation is 
worth doing, other groups than the evaluated organization have 
some interest in finding out about the results. Second, is 
that, given different audiences, several mediums/methods of 
dissemination may be called for. Anderson and Ball classify 
some fifteen different audiences that should at least be con- 
sidered for dissemination of results [Ref. 35] . 

Use of results is another consideration once the 
evaluation is completed. The logical expectation is that the 
decision maker will use the results to make rational future 
decisions. Weiss lists several reasons why that is not always 
the case. The nature of the organization is sometimes opposed 
to the use of the results; however, if the decision maker and 
the evaluator have maintained close communications throughout 
the process, the potential objections can be reduced [Refs. 
35,36]. Particular consideration must be given not only to 
factual evaluation of each program, but also the salesmanship 
aspect of presenting the evaluations. 

Anderson and Ball have adapted, from a paper by J. S. 
Berke a structure which they suggest should be applied when 
writing evaluation reports. 

•Brevity and clarity. Critical findings should be 
summarized clearly and simply at the outset. 

•Timeliness — to be useful and utilized, results must 
be reported according to other peoples' schedules and 
not the evaluator's research clock. 
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•Interim products and reports--these help prepare deci- 
sion makers for the impact of larger, later evaluation 
reports. Besides, they can allow preliminary planning 
for use even before the final report is available. 

•Responsiveness--he notes here that traditional researchers 
tend to make questions more interesting, designs more 
elegant, the analysis more comprehensive, and the use of 
recommendations more guarded than necessary. 

[Ref. 35: pp. 107-108] 

In summary, the program evaluator has the responsibility 
to push the results of the evaluation and to bring them to 
the attention of all those who should be made aware [Ref. 35]. 

4 . Critical Incident Evaluation 



Critical Incident Evaluation Program involves manage- 
ment writing regular reports on the performance they observe. 
The critical incident process has as its roots the idea that 
actual behavior should be appraised, not traits of behavior. 
It is an attempt to justify ratings based on specific inci- 
dents that provide support evidence for the evaluation. 
Levinson places strong emphasis on the need to evaluate on 
how things get done and not just results [Ref. 37]. Levinson 
argues that in reality people are evaluated on the "how", but 
many are led to believe that they are just judged based on 
results . 



Thus, the profit oriented manager pulls his company 
out of the "red", but is criticized for the methods used to 
do it. Levinson cites many deficiencies with current evalua- 
tion systems. One is that no matter how well-defined the 
dimensions for appraising performance on a quantitative basis, 
judgments on performance are always subjective and impression- 
istic [Ref. 37]. Second, delay in giving feedback creates 
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frustration both when performance is good and credit is de- 
served, and when performance is bad and criticism is rendered 
long after the performance. Levinson views performance 
evaluation not as a technique, but a process involving both 
people and data. Thus, the process is inadequate [Ref. 37]. 

Levinson proposes that an effective critical incident 
model be composed of the following [Ref. 37J : 

1. A dynamic job description — one which amplifies 
statements of job responsibility and desired 
outcomes by describing the emotional and 
behavioral topography of the task. 

2. A critical incident process — this requires jointly 
setting objectives and discussing each piece of 
behavior that is judged good or bad by the manager. 

3. A psychological support system — to accomplish 
this, the manager must learn to cope with feelings 
or guilt over the appraisal. He calls this upward 
appraisal concept in which managers who develop 
employees through their effective appraisal should 
be compensated. [Ref. 37] 

The whole critical incident process involves a con- 
tinuous flow of feedback to the evaluated employee. 

Although the critical incident process by its very 
nature should lead to a continuous review and communication 
on positive and negative behavior, that is not always the 
case. Also the critical incident itself is often a subjec- 
tive evaluation on the part of the supervisor. The system 
should focus attention on actual behavior rather than on 
employee traits. Also it provides an opportunity for the 
employee to find out specifically how to perform if that 
individual wants to be rated higher the next time. This 
system does encourage regular feedback since the supervisor 
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can hardly wait to the end of a period to provide the critical 
incident feedback [Ref. 38] . One of the major factors supporting 

this system is that the critical incident process takes the 
surprise out of the annual or semi-annual evaluation. All 
those incidents viewed by the supervisor as critical would 
have already been brought to the attention of and discussed 
with the subordinate. This technique provides more regular 
feedback to the subordinate because both good and bad inci- 
dents should be noted continually. Also if the employee feels 
unfairly judged, that individual may appeal the criticism 
immediately rather than waiting a long period of time [Ref. 37]. 

E. SPECIFIC EVALUATION MODELS APPLIED TO THE MILITARY 

The approaches to the evaluation process discussed thus 
far have been general in nature. However, evaluation proc- 
esses specific to military environments were also reviewed. 
Following are four evaluation approaches to the military 
which were reviewed in detail. 

1 . Army Training and Evaluation Program (ARTEP) 

The Army Training and Evaluation Program (ARTEP) for 
Infantry Battalions was reviewed for this study. A detailed 
brief on ARTEP was provided by the G-3, 7th Infantry Division 
[Ref. 39] . The Army Program specifics are provided for 
Infantry, Airborne, Air Assault, and Ranger units in one 
volume [Ref. 40]. That guide is further supported by local 
guidance at each command. 



41 



As an example of the ARTEP, the Infantry ARTEP is 
discussed. The Infantry ARTEP Evaluation has four purposes: 
(other references exist for ARTEP application to other areas, 
such as Armor and Mechanized units) : 

1. Establish infantry unit training missions with 
specified tasks, conditions, and standards of 
performance for combat-critical missions. 

2. Under simulated combat conditions, train and 
evaluate the ability of the unit. 

3. Evaluate the effectiveness of past training of all 
echelons of the Battalion. 

4. Assess future training needs. (Ref. 40] 

The evaluation is broken into three phases: 

1. Critique of Leaders 

2. Evaluation of Whole Battalion 

3. Written Report 

The ARTEP employs the following philosophy: 

1. The evaluation is conducted two levels down (Division 
evaluates Battalion) 

2. Battalion ARTEP ' s are scheduled every 18 months. 

3. Companies/Platoons/Squadrons are evaluated annually. 

4. Subunit evaluators focus on the end evaluation. 

5. Evaluations are scheduled in sequential progression 
(squad, platoon, company, then battalion) 

6. Scheduled preparation time is five weeks for a 
Battalion 
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7 . 



Fence off the unit for two weeks following the exer- 
cise (no other requirements are placed on the unit 
during this period) 

8. Use operating force tactics 

9. Do not evaluate during other training or field 
exercises. [Ref. 39J 

The ARTEP Evaluations are conducted strictly within 
the divisional unit. The Assistant Division Commander is 
designated "exercise director, " but a sister battalion is 
designated as the force to provide the senior evaluator, 
another lieutenant colonel as well as company and other 
evaluators. The ARTEP is viewed as a training evaluation. 

The ARTEP System provides a combination of both objec- 
tive and subjective feedback to the evaluated unit. All of 
the tasks evaluated in the ARTEP evaluation are reported on 
a Satisfactory/Unsatisfactory basis. All unsatisfactory or not 
evaluated ratings must be explained in the remarks. This 
data is placed on a standardized data collection sheet. A 
copy is provided to the evaluated unit shortly after the 
exercise is completed. Additionally a copy of the data sheet 
is forwarded (without any identifying unit data) to the U.S. 
Army Infantry School, Fort Benning, GA. The data forwarded 
to higher headquarters is for the purpose of future training 
development efforts and for use in improving the ARTEP doc- 
trine, devices, and techniques Army-wide [Ref. 40]. The 
overall evaluation given to a unit is a Satisfactory/ 
Unsatisfactory. No percentile scores are associated with or 
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computed for the ARTEP results. Thus no comparison is made 
from unit to unit using ARTEP data [Ref. 39]. 

In addition to the written report, a continuous review 
and exchange of comments between the evaluator and unit is 
encouraged. Shortly after the ARTEP is completed, an oral 
debrief is held for all key personnel of the battalion and 
the evaluators. In some cases the detailed (good) evaluations 
of units to the squad level are announced to all of the troops 
at formation. This technique is viewed as a positive motiva- 
tion for the small units and their leaders [Ref. 39 J . 

2 . Funk Conceptual Model of Unit Performance 

A recent study by Steven Funk [Ref. 13] attempted to 
model combat unit effectiveness for the U.S. Army. He found 
many of the same difficulties in defining effectiveness that 
this author found in defining readiness. He described effec- 
tiveness as follows: 

Individual evaluators determine unit effectiveness 
based upon some explicit or intuitive constraint of 
what units are supposed to do and how that is achieved. 

Unit effectiveness is determined by evaluating both 
outcomes and processes, and is determined for perform- 
ance on all tasks presented to the unit whether they 
are combat related or not. [Ref. 13: p. 19] 

He interviewed groups of officers and senior enlisted, 
primarily battalion commanders, company commanders, staff 
officers, unit officers, and first sergeants. He also re- 
viewed a wide array of existing predictive models from the 
Army, Navy, and Air Force. He identified early readiness 
studies such as Army Training Study (ARTS) 1977-1978, which 
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concluded that readiness should be viewed from a perspective 
of relationships among personnel, weapons, equipment, re- 
sources, detractors, and incentives. He also reviewed the 
Navy's 1974 report on readiness which stated, "Organizational 
effectiveness implies an evaluative or judgmental process 
against an expectation or standard" [Ref. 13: p. 11]. 

The result of his studies concluded that military 
unit effectiveness is far more readily conceptualized than 
measured [Ref. 13] . Part of the problem of measuring mili- 
tary unit effectiveness is operationally defining readiness 
in a peacetime environment. 

Funk referenced Etzioni who noted that organizations 
attempt to achieve a balanced distribution of resources 
across needs and did not attempt to maximize satisfaction in 
one area [Ref. 13] . Funk concluded that military units in 
peacetime have great difficulty in maximizing effectiveness 
or readiness because they really face not one objective, but 
a whole array of competing priorities all calling for their 
resources. His studies resulted in the proposed conceptual 
model in Figure 3.3. 

This model is referred to as the Unit Performance 
Systems model. It describes behavior as the interrelation- 
ships of technology, formal and informal unit structure, 
perceived unit requirements and priorities, available re- 
sources, unit climate, unit process, cause of evaluation of 
unit actions, and the unit operating environment [Ref. 11]. 
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Figure 3.3 Unit Performance Systems Mode! 
[Ref. 13 : p. 38] 
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3 . Hayes et al., Statistical Model of Combat Effectiveness 



In 1977, a cross-sectional study for the Defense Ad- 
vanced Research Projects Agency was conducted. An effort was 
made to identify through detailed statistical methods key 
factors which contributed to levels of success in 22 combat 
engagements ranging from World War II to Vietnam. The study 
was aimed at determining the effectiveness of infantry battalions 
based on the judgment of experienced Marine Corps officers in- 
volved in these engagements. Multi-variate statistical analy- 
sis was used to examine the data results. The study revealed 
that adaptive behavior by units, that is reaction to the com- 
bat environment, was the single most important discriminator 
between successful and unsuccessful performance. They also 
identified three types of activity which appeared to be 
closely related to mission accomplishment: command and 

planning (strongly related) , supporting fires (moderately 
associated) , and coordination function (relatively weak and 
appeared to require effective command and planning before it 
made a difference) [Ref. 18]. 

This study defined "combat effectiveness" as 

...the ability of a unit to accomplish a military 
mission. As such, combat effectiveness refers to 
performance in a hostile environment. [Ref. 18: pp. 1-2] 

As such the focus on the effectiveness of a military unit is 

placed on outcomes and effectiveness and can only be measured 

by mission accomplishment. The infantry battalion was seen 

as not operating in a vacuum, but rather in a world of 



obstacles and constraints. Thus the adaptability was an 
important element in performance. Besides adaptability, Hayes 
identified a number of key elements which included: 

•maneuver during the action 

•preparatory air, artillery, and naval gunfire 
• communications 

•quality of planning and information as well as others. 

[Ref. 18: pp. 1-18] 

This study was done during the period when MCCRES was 
being developed. One important observation made here with 
regard to feedback was that, although MCCRES is comprehensive 
and standardized, a great deal of information is currently 
passing through the hands of the evaluators that could be 
extremely valuable to the evaluated units [Ref. 18]. 

4 . Sassone Economic Model of Training Effectiveness 
A fourth approach to effectiveness is taken by 
Sassone, who has taken an economic view which is different 
from the traditional economic approach to evaluating military 
training programs. The traditional approach is to evaluate 
the training on its projected costs and benefits. The Sassone 
approach requires that any new training program be compared 
directly with existing programs that it will replace or 
amplify. Traditionally equipment and training are judged 
simply on their ability to function for the intended purpose. 
The training programs are particularly difficult to evaluate 
because of the necessity of measuring the impact of the train- 
ing on the trainee. In the private sector training programs 
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are generally judged on their effectiveness by the change in 
lifetime earnings of the trainee after the program [Ref. 41]. 

If the cost of the training program is less than the lifetime 
increase in wages, then this program is judged to be economi- 
cally worthwhile. However, training programs in the mili- 
tary cannot be judged this same way, because there is currently 
no way to measure their contribution toward readiness , sus- 
tainability, or effectiveness since these are not commodities 
or traded in the market place [Ref. 41] . 

Sassone's methodology requires that when a new type 
of training for an individual or unit is developed, it must 
be stated in specific terms to what extent it substitutes for 
existing programs. The constraint in this case is the train- 
ing budget for the specific type of training. There are 
three steps in this methodology [Ref. 41]. 

(1) An equation relating training inputs and outputs must 
be developed. This requires a comparison of the two types 
of training under consideration (see Figure 3.4). The 
figure shows different levels of effectiveness which can be 
achieved with various combinations of the two types of train- 
ing. The curve shows all the levels of effectiveness which 
can be achieved with the various combinations. The shape of 
the curves describe the substitute ability of one training 
type for the other. 

(2) Relationship of cost and use data must be obtained. 

There are three types of costs involved: front end costs for 
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Figure 3.4 A Hypothetical Training Input vs. 
Output Relationship. 
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the training program under evaluation; operational costs, and 
training budgets for the training manager of the program. 

(3) The last step requires the development of a relation- 
ship between training and effectiveness. 

Where potentially greater effectiveness is available 
at greater cost, the issue is simply whether the 
greater effectiveness is worth the greater cost. 

[Ref. 41: p. 41] 

Sassone fails to specify how the relation of training and 
effectiveness is obtained. It is assumed in this study that 
the relationship is identified by some evaluation method such 
as MCCRES . Measures such as those generalized by ARTEP allow 
for no relative measure of effectiveness which is required for 
this type of comparison. 

The cost-effectiveness equation is then developed 
by optimum use of available training resources at each level 
of the training budget. The term "optimum" in Sassone' s 
proposal is one that seems to be in conflict with other 
theorists. Funk [Ref. 13] for instance indicates that many 
competing priorities take resources from the unit, thus 
precluding optimality of resource application. 

Figure 3.5 represents a relation between cost and a 
sample program. Figure 3.5 is described by Sassone as starting 
at a minimal level, indicating that some level of effective- 
ness would exist even without training. The curve initially 
rises rapidly but starts to level off as maximum effectiveness 
is reached. As better training programs are conceived, the 
middle of this curve would shift up. If some new program were 
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curve after a new training 




A Budget Level 

Source 1 Adapted from ^Ref. 41= p. 41 j 

Figure 3.5. The Military Value of a 
Training Program. 
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a perfect substitute of an existing one, the new curve would 
overlay the existing one. Upward shifts in the curve can be 
caused by a greater level of effectiveness achieved for a 
lower cost. The Figure 3.5 shows how the military value of 
various training programs can be compared with other or exist- 
ing programs. Additionally, one must consider opportunity 
costs, funds used for research and development of new pro- 
grams which the service could otherwise have devoted to 
existing programs [Ref. 41] . 

The military value of the new training program is repre- 
sented by the increment in the current training budget 
needed to increase effectiveness by the same amount 
as the increase in effectiveness associated with the 
new program. [Ref. 41: p. 42] 

The opportunity cost of a new military training program is 

determined by spreading the front end costs equally over 

the units trained per year and the expected life time of the 

program applying the appropriate discount rate. Figure 3.5 

is used to compare the military value of a training program 

by using the cost-effectiveness ratio, if the military value 

is less than the opportunity cost, then the new program would 

not be considered economically feasible [Ref. 41]. 

5 . Summary 

This section reviewed four approaches to evaluation 
that have recently been developed for applications to the 
military. The ARTEP employed [Ref. 40] by the Army provided 
a process of evaluation which has much in common with MCCRES 
especially in its application to similar organizations. The 
conceptual model by Funk [Ref. 13] provided a view not only 
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of the elements involved in training, but also of some of 
the barriers and constraints that units must face in a peace- 
time environment. The results of the Hayes et al., Statistical 
Model [Ref. 18] were valuable in that specific characteris- 
tics were identified that led to success in actual combat 
engagements. Finally the Sassone, Economic Training Model 
was presented [Ref. 41] which proposed some new thoughts on 
the subject of evaluating military training programs. The 
ideas brought forth in these studies are of use in the examina- 
tion of MCCRES in that they provide a rich background against 
which MCCRES can be viewed. 

F . FEEDBACK 

...studies have shown that accurate feedback about 
quality of work is a strong stimulus to good perform- 
ance. . .People work better when they know how well they 
are doing in relation to some meaningful standard. 

[Ref. 42: p. 174] 

This final section of the chapter discusses feedback. 
Feedback is variously described by different writers as feed- 
back [Refs. 27,29,37,43], feedback control [Ref. 16], 
communicat ions [Ref. 34] and further as dissemination, 
communications, and utilization [Ref. 35]. Whatever the 
feedback is called, it is common to all of the models dis- 
cussed. Feedback is a necessary part of all evaluation 
systems . 

Control to be effective, requires timely, accurate, and 
dependable indicators of effectiveness. Feedback should 
provide information that is adequate to suggest appro- 
priate action. To that end, reports must point to 
significant developments, as distinguished from what is 
normal, usual, and to be expected. [Ref. 32: p. 504] 
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Flamholtz [Ref. 44] has presented a typical model, 

Figure 3. 5, of an organizational control system which places 
emphasis on the feedback loop. He describes feedback as a 
necessary managerial function of the control systems which 
identifies problems [Ref. 44]. 

This model provides an example of how the essential feed- 
back loop functions in an organizational control or evaluation 
system. The feedback loop provides an interactive process 
that allows for routine recycling of the results of a compari- 
son of the observed performance to the measurement system. 

The essential element to observe here is that feedback is a 
standard fixture of the evaluation system. All systems viewed 
that provide a diagrammatic model include a feedback 
mechanism. In addition Anderson and Ball and the ARTEP 
provided detailed structures for feedback. 

G. SUMMARY 

This chapter has covered the topics of management, manage- 
ment control, evaluation, and feedback. These topics can be 
seen as moving from the general to the specific. The bound- 
aries between these topics are not clear cut. There is a 
great deal of overlap in the fields of management, organiza- 
tional control, evaluation, and feedback. This is particu- 
larly true when they are viewed from different disciplines. 

For instance, much of what is classified in education as 
evaluation, would be classified under control in organizational 
or managerial theory. A similar situation exists with regard 
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Source 1 Adopted from Ref. 4 4 p. 
Figure 3.6 Mode! of Organizational 
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to terminology for feedback. Theorists in the same and 
different disciplines often use various terms to describe 
feedback . 

Feedback was discussed with emphasis on relating the 
many different terms used to describe feedback. in addition 
the feedback loop which is common to all models reviewed 
has been discussed. The theory of feedback is essential to 
this study and the many ideas and structures placed on it 
will give much insight when applied to MCCRES. Certain ele- 
ments such as: brevity and clarity, timeliness, interim 

products and reports, and responsiveness [Ref. 35] have almost 
universal application to feedback theory. The MBO , Evaluation 
and Authority, Critical Incident, and ARTEP all provide a 
wealth of insight on how feedback techniques should be 
applied . 

The next chapter provides a detailed overview of the 
MCCRES System. 
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IV. MCCRES 



In the last chapter a detailed examination was made of 
management control, four general evaluation systems, and four 
specific evaluation systems which apply to the military. 

This should provide the reader with a background against 
which to view MCCRES. In this chapter attention is given in 
detail to the MCCRES itself, its development, explanation of 
the system's mission performance standards, and the evalua- 
tion process applied by MCCRES as well as the reports and 
feedback designed into the current system. The goal of this 
chapter is to provide the reader with a general understanding 
of how the MCCRES is applied, based on the most current 
Marine Corps directives. 

The purpose of the Marine Corps Combat Readiness Evalua- 
tion System (MCCRES) is to provide a timely and accurate 
evaluation of readiness of Fleet Marine Forces, includ- 
ing reserve units, to accomplish assigned missions. 

[Ref. 1: p. 1-A-lj 

The formal MCCRES must be given at least once every two years 
for all FMF units for which performance standards have been 
written. 

A. DEVELOPMENT AND STRUCTURE 

The development of the MCCRES was begun with a review of 
the various combat readiness measurements then currently in 
use. Many of these consisted of individual, as well as unit, 
evaluations and were in the form of inspections which stressed 
appearance more than combat proficiency [Ref. 1]. 



53 



The MCCRES was structured to access the ability of all 
Marine Corps Combat units: air and ground combat, combat 

support, and combat service support. Specifically it was 
designed to provide: 

-Performance standards based on assigned missions 
-A standardized evaluation process 
-A standardized reporting system 

-Feedback to units indicating strengths and weaknesses. 

[Ref. 1: p. l-A-2] 

The MCCRES was designed to provide a detailed analysis 
of the units' operational capabilities and to specifically 
identify strengths and weaknesses through comparison with 
doctrine under simulated combat conditions. The doctrine is 
embodied in a checklist of requirements. The hierarchy of 
these qualities is identified from general to specific as 
Mission Performance Standards (MPS) , each of which consists 
of three parts: Tasks, Conditions, and Requirements. The 

MPS set the Commandant’s acceptable standards for tactical 
performance throughout the Marine Corps. Based on compari- 
son of the unit's performance to the MPS under simulated 
conditions, a determination, is made of "Combat Ready/Not 
Combat Ready." The MCCRES is designed for both formal and 
informal evaluation. The informal evaluation can be pro- 
vided by the unit itself measuring its performance standards. 
The formal evaluation is accomplished by higher- level commands 
normally evaluating a selection of MPS. For the formal 
evaluation certain MPS are required in all evaluations and 
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others are selected by the organization directing the evalua- 
tion based on detailed guidance in Ref. 1. A cross section 
of MPS's are used to provide a statement of the organiza- 
tion's operational readiness. 

The evaluators employed in the formal system must be ex- 
perienced, capable personnel. The overall system is designed 
to identify specific operational deficiencies and the nature 
of any latent or potential problems. The MCCRES cycle is 
described in Figure 4.1. 

B. MISSION PERFORMANCE STANDARDS (MPS) 

In building the MCCRES framework it was decided that the 
evaluation would be structured around a set of Mission Per- 
formance Standards that establish, on a Marine Corps-wide 
basis, acceptable operational performance. Derivation of 
these standards was accomplished by using five interrelated 
concepts concerning determination of combat readiness. The 
concepts are: 

- Standards must be objective 

- Standards must define for the evaluator what quality 
means 

- Standards must be based on published doctrine 

- Standards must involve the performance of individual 
Marines in evaluation of unit combat readiness 

- Standards must be simple for the evaluator. [Ref. 1] 

The MPS are defined in detail in Volumes II through VIII 

of Marine Corps Order 3501.2. Each of these volumes pertains 
to a specific type of unit to be evaluated, for instance 
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Source : Adopted from [Ref. 1 : p. I - A — 6 J 
Figure 4.1. MCCRE5 Cycle 
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Volume II pertains to Infantry and Volume III to Rotary Wing 
Observation Squadrons. In this way the volume lists the 
MPS's involved for each unique type of unit. 

Each MPS is further broken down into three major parts: 

-The Task to be performed 

-The Condition under which the task is to be performed 

-The Requirements which must be accomplished to success- 
fully fulfill the task. [Ref. 1: p. I-B4J 

Examples of the MPS, Tasks, Conditions, and Requirements 
are provided in Figure 4.2. 

To assist the Evaluator when the Requirement does not 
completely define the quality of performance necessary. Key 
Indicators (KI) are provided. These KI's are defined to 
clarify and explain the requirements [Ref. 1] . Samples of 
Key Indicators are presented in Figure 4.3. 

For simplicity and to reduce the need for subjective judg 
ments , all MPS's were designed to permit the Evaluator only 
three possible outcomes for each requirement: Yes/No/Not 

Applicable. Those MPS's dealing with the performance of indi 
vidual Marines, Exercise of Command and Control, and Fire 
Support Coordination must be used during every evaluation. 

The reason that all MPS's are not employed during each 
evaluation is a limitation of time and resources [Ref. 1J . 

In addition to the evaluation of MPS's which use a simu- 
lated combat environment against an aggressor force, a series 
of MCCRES Standard Performance Tests (SPT) have also been 
developed. Examples of these types of proficiency tests are: 
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MISSION PERFORMANCE STANDARD 2B.3— MOVEMENT TO CONTACT 



:s 

RATION 



iource: 



CONDITIONS REQUIREMENTS 



Y 



Operation order is 2B.3. 

provided directing a 
tactical movement to 
area of anticipated 
contact with enemy. 2B.3. 

Route selection is a 

tactical decision of 

unit commander. 2B.3. 

Minimum preparation 

time for proper 

evaluation of this 

task is three hours 

after delivery of 

the order to the 2B.3. 

unit. 



.1— Receipt of order 

acknowledged to higher 
hg. 

.2— Warning order issued to 
all subordinate units 
within 30 minutes (Kl). 

.3— Staff coordination 

I AW FMFM3-1; Emphasis: 
Intelligence collection, 
fire support planning, 
and security (Kl). 

(Key Indicator) 

.4— All echelons prepare for 
move; inspect material, 
pack equipment, and iden- 
tify specialized 
requirements. 



2B.3.1.5 — Weapons test fired if 
situation permits. 

2B.3.1.6 — Communications checks com- 
pleted and communication 
security materials issued. 

2B. 3. 1.7— Movement plan formulated (Kl). 



2B. 3. 1.8— Movement order issued (Kl). 



2B.3.1.9 — Readiness for movement re- 
ported to higher hq. 

Adopted from (Ref. 1: 
p. 1-B-5) 



Figure 4.2. Sample of Mission Performance Standards 



N COMMENTS 
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WARNING ORDER 



MUST INCLUDE: 

General information on the situation. 

Units to make the move and anticipated sequence. 
Anticipated time of move. 

Anticipated route and destination. 



STAFF COORDINATION 

MUST SPECIFICALLY ADDRESS: 

Route reconnaissance — map, physical, or aerial photo. 

Support needed from higher hq — air, vehicle, or firepower. 

Fire support coverage throughout movement. 

Cover and concealment available enroute. 

Logistic aspects affected by location change. 



BASIC PLAN 

MUST BE BASED ON: 

Movement at speed desired by higher hq. 

Control of all elements during move. 

Security of all movement serials. 

Known enemy capabilities. 



MOVEMENT ORDER 

MUST CONTAIN: 

Clearly stated mission. 

Definition of all control measures to be used: check points, phase lines, march objectives, etc. 
Specific missions for attached and/or support elements: tanks, engineers, TOW, etc. 
Identification of initial point from which move will begin. 

Assignment to march serials. _ . 

Time of departure. 

Clear identification of available outside support. 

Detailed security procedures. 

As much information on threat as is available: 

Emphasis on specialized weaponry tnat can affect the move: ATGM, artillery, air, etc. 



Source: Adopted from (Ref. 1.P: p. I-B-6) 

Figure 4.3 Sample of Key Indicators for Evaluator 
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Foot Mobility Test, Dragon Gunner's Test, Engineer Route 
Reconnaissance Test, and Aircraft Recognition Test. The 
SPT ' s are used to support the MPS ' s and are tested objectively 
prior to the MCCRES operational/readiness test. The SPT is 
formated exactly as the MPS; however, a 'Yes' must be achieved 
in each area before the MCCRES is continued. Any unit not 
found proficient in the SPT portion of MCCRES will not be 
tested on the tactical portion. 

C. PERSONNEL ASSIGNMENT FOR THE EVALUATION 

The key to successful application of each MCCRES is the 
evaluation process. Selection and training of evaluators is 
at the very heart of the evaluation process. Credibility of 
the evaluators is essential to the success of MCCRES. The 
evaluators selected are to have the requisite skills and 
recent experience as to preclude the need for a long, detailed 
school for the evaluators. However, as needed and as time 
allows, training is provided to evaluators. Since all evalu- 
ation systems require some judgment on the part of the 
evaluators, and MCCRES is no exception, sound judgment is a 
prerequisite for each evaluator chosen. 

The evaluation is structured with the Commanding General's 
Fleet Marine Force Pacific (FMFPAC) and Fleet Marine Force 
Atlantic (FMFLANT) as the Evaluation/Exercise Commander (s) 

(EC) for the initiation and conduct of all formal MCCRES 
Evaluations. The responsibilities of the Evaluation/Exercise 
Commanders are outlined in detail in Ref. 1. 
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1. Evaluation/Exercise Director (ED) — is designated by the 
Exercise Commander (EC) and is responsible to prepare 
for, conduct, and report the formal MCCRES evalua- 
tions. Normally the Commanding General of the respec- 
tive Marine Division or Wing whose subunit is to be 
evaluated is designated ED. 

2. Tactical Exercise Controller (TEC) — is designated by 
the ED, along with an appointed staff, to serve as the 
control agency for conducting the exercise. The TEC 
compiles and analyzes the results of the evaluation. 

The TEC is also responsible for the detailed training 
of the evaluators as well as development of a detailed 
exercise scenario. At the end of the evaluation a 
formal report is prepared for the ED describing the 
combat readiness of the evaluated unit. A detailed 
critique should also be conducted for all involved in 
the evaluation. In the critique the results of the 
evaluated MPS are to be highlighted with respective 
strengths and weaknesses. 

3. Evaluators — should be prepared for the role through 

successful past professional experiences as well as 
through any detailed school provided. The MCCRES 
evaluators have three roles: exercise controller, umpire 

and performance evaluator. As a performance evaluator, 
they apply the detailed MPS's contained in the appro- 
priate volume for the unit and they are evaluating and 
actually make the 'Yes/No/Not Applicable' determinations 
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for individual tasks. The evaluators must possess a 
complete and thorough understanding of the Mission 
Performance Standards being evaluated. The 
evaluators must make any notes needed on an Evaluator 
Work Sheet that are necessary to support judgment 
during the process. [Ref. 1] 

4. Senior Evaluator — determined by rank. Evaluator for 
the unit evaluated compiles the data sheets from all 
evaluators and should conduct a post exercise wrap up. 

Any questions or conflicts should be resolved at that 
time. The process must result in the senior evaluator 
being assured that his data provides an accurate reflec- 
tion of the overall unit performance. The senior 
evaluator then makes the determination of the overall 
unit evaluation of "Combat Ready/Not Combat Ready." 

The senior evaluator provides the data sheets to TEC 
who compiles the detailed analysis and presents it to 
the evaluation/exercise director [Ref. 1] . 

5. Evaluation Staffing — is the responsibility of the ED 
for both selecting and training all evaluators prior 
to a MCCRES . It is desirable to obtain evaluators 
from adjacent commands, not directly related to the 
organization being evaluated. A more complete discussion 
of providing the evaluators can be found in Ref. 6. 

D. COMPUTER ADAPTED MCCRES 

The Marine Corps Combat Readiness System Software Appli- 
cation (MCCRESSA) , is designed to furnish all organizations 
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involved with MCCRES a means of assessing and analyzing data 
pertaining to MCCRES evaluations. Specifically it provides 
the following: 

A list of mission performance standards, tasks, and 
requirements . 

...the chief evaluator with a rapid overview and selected 
analysis of the units evaluation. 

Identifies unit readiness based upon mission performance 
standards tested within the applied scenario. 

Provides unit commanders a list of unit readiness defi- 
ciencies to initiate corrective action or obtain 
higher command level assistance. 

Reviews unit evaluations to determine validity of mission 
performance standards and doctrine. 

Reviews unit's evaluations to determine limitation/inade- 
quacies in officer education programs. [Ref. 1: p. I-E2] 

All of the formal evaluation reports are input to the 
Headquarters Marine Corps data base [Ref. 1]. 

E. REPORTS 

At the end of a MCCRES, after the Evaluator Data Sheets 
have been compiled for all units evaluated, the TEC and 
senior evaluators should review the sheets and make a deter- 
mination of "Combat Ready/Not Combat Ready" for each unit 
evaluated. This recommendation is based on an initial review 
of "Yes's/No's" for all requirements, taking into account 
any tasks or MPS ' s that were "demand elements that must be 
judged Yes for an overall 'Combat Ready' evaluation to be 
given . " 
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1. Feedback 



"The primary purpose of MCCRES reports is to provide 
the feedback necessary for commanders to initiate corrective 
action that will improve combat readiness" [Ref. 1: p. 1-Dl] . 

The results should be reflected in improved training 
objectives and may also affect resource allocation of per- 
sonnel, equipment, or logistics support. To assist in the 
reallocation of critical resources, a detailed report is 
provided through the chain of command to the unit evaluated. 
This report provides a short subjective comment as well as 
percentile scores and weighting for each Section, MPS , Task, 
and Requirement evaluated. (See Appendix A — Sample MCCRESSA 
printout for one MPS.) 

Additionally a report is provided in message format 
to the Commandant of the Marine Corps (CMC) within 10 days 
following the end of an evaluation indicating the overall 
evaluation "Combat Ready/Not Combat Ready" and unit identifi- 
cation. Within 30 days a detailed report of the MCCRESSA 
printout is provided to the CMC. The report should include 
comments or recommendations for improvement or revisions to 
the MCCRES. The purpose of the follow up report is to allow 
CMC to: provide assistance in the review of doctrine, tactics, 
techniques, education and training programs, and validation 
of MCCRES elements . This report also serves to highlight 
trends and repeated deficiencies, and permit analysis for 
corrective action to take place [Ref. 1] . 
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2 . Policy 



Based on Marine Corps policy, MCCRES reports are not 
designed to be used to compare the combat readiness of vari- 
ous units. This is a result of differences in unit type, 
services, environmental conditions, and resource allocations. 
The evaluation indexes on MCCRES reports are provided to 
indicate an approximate status of the unit's combat readiness 
during a particular evaluation [Ref. 1] . It can be said that 
the MCCRES provides a snap shot of the unit at the time of 
the evaluation. 

F. SUMMARY 

This chapter presented an overview of the MCCRES including 
a discussion of the Development and Structure, Mission Per- 
formance Standards, Personnel Assignment for the Evaluation, 
Computer Adapted MCCRES and Reports . The purpose was to 
provide the reader with a general background of how the 
MCCRES is intended to be implemented throughout the Marine 
Corps, based on systems design and policies. 
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V. METHOD OF ANALYSIS 



The research method used had three distinct phases. 

First, a general literature review was conducted. Second, a 
specific literature review of MCCRES and related defense pub- 
lications was undertaken. Third, a detailed field study was 
made to obtain first hand information from those involved 
with the development, application, and oversite of MCCRES. 

A. AUDIENCE 

Anderson and Ball discussed [Ref. 35] the importance of 
identifying the audience when researching and distributing 
results. To this end the primary audience for this study 
are the planners who conduct periodic reviews and revisions 
of MCCRES doctrine, that is, the Readiness Branch at HQMC. 
Additionally the audience should include all those involved 
with MCCRES or who will be involved with MCCRES, as well as 
commanders, evaluators, planners in organizations and others 
who would benefit from understanding the findings presented 
herein . 

B. GENERAL LITERATURE REVIEW 

The results of the general literature review are presented 
in Chapter III. A background in management control, general 
evaluation systems as well as specific examples of evaluation 
systems applied to the military provided a backdrop against 
which MCCRES can be viewed. It is evident from Chapter IV 
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that MCCRES has incorporated into it many of the positive 
aspects of general evaluation systems theory [Refs. 5,7,22, 
27,28,34,35] . 

C. SPECIFIC LITERATURE REVIEW 

The specific literature review consisted of articles on 
MCCRES appearing in the various defense publications [Refs. 2, 
7,11,14,33,39, and 40] as well as the technical reports and 
articles available on MCCRES and other military training and 
readiness evaluation systems [Refs. 4,8,12,13,18,40]. This 
specific literature review not only provided a basis for 
understanding the MCCRES from the standpoint of its proponents 
and detractors, but it also provided some understanding of 
alternatives to MCCRES. The detailed reports made available 
by George Washington University were invaluable in understand- 
ing the background and initial development of the MCCRES 
[Refs . 3,4, 11] . 

D. FIELD STUDY 

The field study was conducted to obtain first hand infor- 
mation on MCCRES. Personal interviews, telephone interviews, 
and a visit to the Readiness Branch HQMC, George Washington 
University, and to the Army's 7th Infantry Division were em- 
ployed to that end. The interviews were semi-structured. 

The stated purpose of the interviews was not just to answer 
the specific questions posed, but to stimulate thought and 
identify as many ideas as possible on how the MCCRES could 
be improved. Although the scope of this project was limited 
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by its primary emphasis on feedback, other valuable informa- 
tion was elicited. 

1 . Sample 

The sample group chosen consisted of those who had 
the most experience and most vested interest in the MCCRES 
feedback. Because the MCCRES evaluations are most often 
given to infantry battalions and aviation squadrons, these 
were identified as the primary target group. Battalion and 
squadron commanders, senior evaluators, and division/squadron 
MCCRES officers were identified as the primary sources of 
information since they have the most detailed involvement with 
the MCCRES feedback. Upon interviewing the sample it was 
also found that many of the officers had been involved in 
several MCCRES 's and in different capacities. Thus many had 
experience as both a senior evaluator and a battalion or 
squadron commander of an evaluated unit. Divisional or 
wing MCCRES officers were also a rich source of information 
because of their daily involvement with MCCRES. The secondary 
source of information were "others involved with MCCRES." 

This group is composed of members of evaluated units other 
than the unit commanders, such as unit operations officer, 
officers involved with subunits evaluated, and company 
commanders. They also provided much information. 

Given the sample identified, the field study, there- 
fore, concentrated on those of rank from captain to colonel 
in the military occupational fields of infantry, aviation 
(fixed wing and helicopters), tanks, artillery, engineer, 
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and air control. A total of 37 individuals were interviewed. 
The sample group is presented by rank and occupational field 
below in Table 5.1. 




Major 34 3 1 0 0 0 11 

Captain 2 0 0 10 0 0 3 

TOTAL : 18 8 6 2 1 1 1 37 

These officers are currently serving in billets throughout 
the Marine Corps, representing all divisions and wings, as 
well as the 1st Marine Brigade in Hawaii. Other active duty 
officers serving in billets’ at Headquarters Marine Corps, 
Naval Postgraduate School, and with the reserve establishment 
were also interviewed. 

2 . Interview Techniques 

All those interviewed were asked a standard set of 
open-ended questions [Ref. 45]. These questions were pur- 
posefully left broad and open-ended to solicit as many ideas 
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and opinions as possible. Each interview was conducted 
in person or by telephone by one of the researchers. All 
interview results were recorded on an answer form and the 
interviews were taped after the permission of the respondent 
was given. When a respondent was uncertain or uncomfortable 
in answering a question, he was asked to skip over it. How- 
ever, no attempt was made to limit the respondent to any set 
of prearranged answers. In all cases, where answer choices 
were posed, an invitation for "other ideas or suggestions" 
was made. As the interviews progressed, the questions were 
refined to help the interviewee narrow in on the specific 
area being researched. 

3 . Questions Posed 

The following set of questions was posed to all those 
interviewed: 

(1) When the computerized data results from a MCCRES are 
provided to the evaluated unit, what uses are made 
of that data? 

(2) Do you feel that information/data provided to the 
evaluated unit as the result of a MCCRES are satis- 
factory for the purpose of initiating corrective 
action that will improve combat readiness? 

(3) Is the current procedure of providing the results as 
a computer printout based on the evaluation of the 
Sections, MPS , Tasks, and Requirements the most 
effective format for displaying the results to the 
evaluated unit commander? 



75 



( 4 ) 



Does the format lend itself to a complete understanding 
of how the unit performed compared with MCCRES 
criteria? 

(5) Do Marine Corps Directives provide sufficient guidance 
for interpretation of results? 

(6) Would it be helpful to know how your unit performed 

on a Section/MPS/Task/Requirement as compared with 
other units? Example: Your infantry battalion scored 

a 73% on the MPS , "Command and Control Operations." 

This compares with a median of 68% for all other 
infantry units and 71% for all units evaluated. 

(7) What do you feel is the most effective forum for pro- 
viding MCCRES results to the unit tested? 

a. Oral debrief at end of exercise 

b. Written debrief at end of exercise 

c. Progressive oral debrief throughout the exercise 

d. Combination of a-c above 

e. Some other method than those suggested 

(8) Who should be made aware of the MCCRES results? 

a. Chain of command 

b. Other like-units (infantry, air, artillery, etc.) 

c. All Marine Corps units 
4 . Background Visits 

Additionally visits were made to Headquarters Marine 
Corps (HQMC) , Readiness Branch, George Washington University 
and to the Army's 7th Infantry Division, Fort Ord, California. 
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Individuals interviewed during these visits provided many 
valuable insights into the development and overseeing of 
MCCRES , as well as a detailed understanding of how the Army 
ARTEP system is employed. Several of the individuals inter- 
viewed were involved in the early development of the MCCRES: 
Professor W. H. Marlowe, George Washington University; Lt. 
Colonel Paul Catalone, USMC, and Colonel M. P. Sullivan, 

USMC (telephone interview) , and Lt . Colonel R. S. Gibson, 
USMC (telephone interview) . 

E. CONTENT ANALYSIS/DATA REVIEW 

Since the material/responses to the questions asked were 
subjective in nature, content analysis was used to analyze 
the results [Ref. 46]. 

1 . Content Analysis Defined 

Berelson states: "Systematic content analysis 

attempts to define casual descriptions of the content, so as 
to show objectively the nature and relative strength of the 
stimuli applied to the reader or listener" [Ref. 46: p. 14]. 

This procedure is employed by constructing an analy- 
sis outline which, as described by Cartwright, embodies the 
following general principles : 

Step 1 — specify data needs. 

Step 2 — map out plans for tabulation. 

Step 3 — map a skeleton of the outline, 

Step 4 — fill in categories for each variable, 

Step 5 — establish procedures for using the material, 
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Step 6--try out the analysis outline and use procedure. 

[Ref. 45: pp. 454-460] 

The application of content analysis allows the re- 
searcher to study the communications and focus on the inter- 
action through messages which link communicating parties. 

Not all significance can be extracted by inspection or mere 
observation. The real purpose of any analysis is to illuminate 
and make inferences about something that is not otherwise 
apparent. Thus the process of content analysis is a process 
like that used when applying the congruence definition of 
evaluation which employs objectives and criteria in develop- 
ing a measurement process which is both scientific and objec- 
tive. The process is not, however, totally devoid of judgment 
and does allow the researcher discretion in categorizing 
results [Ref. 47]. 

2 . Analysis of Specific Responses to Questions 

The purpose of content analysis is to provide some 
means of quantifying subjective information. To accomplish 
this rules must be established. The data contained in the 
response to each of the eight questions was treated as a 
separate unit and no attempt was made during the interview to 
relate any answers from one question to that of another. This 
procedure was used to solicit as much information possible 
and not to limit any of the respondents in their comments. 

Since each question stood alone and was relatively subjective 
in nature, each question provided an opportunity for a response 
to contain similar information as that given to different 
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questions . Thus once a respondent began to provide informa- 
tion to a question which was more appropriately categorized 
as a response to a different question, the interviewer simply 
recorded the information under the question currently being 
asked. However as part of the analysis of the interviews, 
responses to one question that were relevant to other ques- 
tions were considered. 

Each question had certain categories set up for the 
responses and each response made was placed in one of these 
categories. In order to generate useful information from 
content analysis, the categories must be collectively exhaus- 
tive and mutually exclusive. For example: 

Question #1--When the computerized data results from a MCCRES 
are provided to the evaluated unit, what uses are made of 
that data? 

Responses to this question fell into three groupings: 

"none" 

"used for planning future training and resource allocation" 
"historical reference only" 

It should be understood that each MCCRES has many key 
participants and each participant has a different involvement 
in the evaluation. Seme such as the unit commander and senior 
evaluator are involved virtually in all aspects. Others such 
as the operations officers and operations evaluators have a 
more limited focus in their responsibilities. Thus a system 
was developed to give varying weight to those who had differ- 
ing levels of involvement with MCCRES . 
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A scoring system was established with the following 
weighting : 

Intensity Points 

High 3 

Medium 2 

Low 1 

High Intensity — those involved directly in one or more 

MCCRES as unit commander, division or 
wing MCCRES officer, senior evaluator. 

Medium Intensity — those involved with less direct responsi- 
bility for results: operations officers, 

other evaluators, company commanders, etc. 

Low Intensity — Employed in only one case where interviewee 

was involved in a forerunner evaluation to 
MCCRES 

3 . Analysis of General Responses to Questions 

To determine which question responses should be dis- 
played for the reader, a 10% rule was formulated. If 10% or 

more of the respondents to a question amplified their 

' 

specific answer with similar comments or suggestions, then 
the comments were included. Thus, any of the general responses 
included in the next chapter were mentioned by four or more 
of those interviewed. 



F. SUMMARY 

In this chapter the research method is summarized. A 
total of 37 Marine officers were interviewed. Although the 
number is relatively small, it should be remembered that the 
Marine Corps has three infantry divisions and three aircraft 
wings on active duty. Therefore, the total population of 
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those qualified to answer the questions is drawn from a 
limited population of officers in the target group. It is 
difficult to estimate the total number of officers that have 
been involved in these primary capacities since the billet 
holders rotate regularly; but it is safe to say that only a 
few hundred officers have served as battalion/squadron com- 
manders or senior evaluators of evaluated units. Since the 
MCCRES is only required to be given every two years, the 
sample used appears to be reasonable. 
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VI. RESULTS 



This chapter presents results obtained through the field 
study. The specific and general responses to questions posed 
were content analyzed. The basis for this type of analysis 
is provided by Berelson [Ref. 46J . 

Percentages derived from the content analysis are the 
weighted averages of the points for each category as compared 
with total points for each question. The responses are 
weighted based upon the level of involvement of the respondent 
in previous MCCRES's. In cases where some interviewees did 
not answer a certain question, the percentage is based only 
on the results of those who did answer. No attempt was made 
to categorize reasons why those that failed to respond did so. 

A. ANALYSIS OF SPECIFIC RESPONSES TO QUESTIONS 

First Question: (1) When the computerized data results 

from a MCCRES are provided to the evaluated unit, what uses 
are made of that data? 

Responses to this question were grouped into three cate- 
gories--None , Used for planning, and Historical reference 
only — and are shown in Table 6.1. The majority of the responded 
(63.6%) said that the results of MCCRES were used for planning 
future training or specifically for future resource allocation. 

Second Question: (2) Do you feel that inf ormat ion/ data 

provided to the evaluated unit as a result of MCCRES is 
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Number Responding: 34 Response Rate: 34/37 x 100% - 91 . 9 % 
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Results of Question One 



satisfactory for the purpose of initiating corrective action 
that will improve combat readiness? 

The responses to this question are categorized as posi- 
tive or negative and are shown in Table 6.2. The majority 
of the respondents (81.3%) said that the information/data 
provided from a MCCRES is satisfactory for initiating correc- 
tive action that will improve combat readiness. 

Third Question: (3) Is the current procedure of pro- 

viding the results as a computer printout based on the results 
of Sections, MPS , Tasks, and Requirements, the most effective 
format for displaying the results to the evaluated unit? 

The responses to this question were categorized as posi- 
tive or negative. As shown by Table 6.3, 64.6% of the 
respondents said that the current procedure of providing the 
results as a computer printout was not the most effective 
format for displaying the results to the evaluated unit. 

Fourth Question: (4) Does the format lend itself to a 

complete understanding of how the unit performed compared with 
MCCRES criteria? 

The response to this question was categorized as positive 
or negative. As shown in Table 6.4, 55.6% of the respondents 
said that the current format lends itself to a complete under- 
standing of how the unit compared with MCCRES criteria. 

Fifth Question: (5) Do Marine Corps Directives provide 

sufficient guidance for interpretation of results? 

The responses to this question were categorized as posi- 
tive or negative. As shown by Table 6.5, 72.9% of the 



84 



cr 



cr 

cd 



33 

cd 

co 

T 3 

O 

3 

G. 

3 

<o 



03 

cn 



33 

CD 

C/> 

X 

O 

3 

CO 

0 > 

33 

1 ) 

cd 



CD 

C/1 



O 

H 

> 

r“ 



Z 

cd 

(Q 

aj 

< 

CD 



0 ) 

13 

o 

co 



< 

0 ) 



33 
m 
c o 

"D 

o 

z 

co 

m 



cn 



C 3 

03 



cn 



03 



03 




cn § 
o o 
9 <£,' 
n> £ 
cd 

CL 



X 

( 5 ’ 

3- 



ro 

oo 



03 



ro 

CD 



X 




3 

Cl 

ro 

3 



CO 

to < 
n o 

5 < 5 ’ 

CD £ 

ST 

CL 



2 

cd rn 

9 * z 

c CO 



-< 



CD 



o 

o 

o 



o 



o 




c/> 

§ 

o o 

2<o* 

CD ^ 

(7 

Cl 



o 

$ 



CD 



03 



-o 

m 

33 

o 

m 

Z 

H 

> 

o 

m 

co 



35 



Results of Quesiion Two 



ponding: 30 Response Rate: 81.1% 
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Results of Question T 



Number Responding: 27 Response Rate: 73.0% 
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TABLE 6.4 



Number Responding: 27 Response Rate: 73.0% 
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TABLE 6 5 



respondents said that Marine Corps Directives provide suffi- 
cient guidance for interpretation of results. 

Sixth Question: (6) Would it be helpful to know how 

your unit performed on a Section/MPS/Task/Requirement as 
compared with other units? Example: Your infantry battalion 

scored a 73% on the MPS , "Command and Control Operations." 
This compares with a median of 68% for all other infantry 
units and 71% for all units tested. 

The responses to this question were categorized as posi- 
tive or negative. As shown by Table 6.6, 55.4% of the 
respondents said that it would be helpful to know how their 
unit performed on a Section/MPS/Task/Requirement as compared 
with other units. 

Seventh Question: (7) What do you feel is the most 

effective forum for providing MCCRES results to the unit? 

This question was evaluated based on selection of one of 
five responses: oral debrief at end of exercise; written 

debrief at end; progressive debrief; combination of above; or 
other. As indicated in Table 6.7, all thirty-seven inter- 
viewees responded to the question. Of the total, 91.5% 
responded that a combination of oral and written debriefs 
is the most effective forum for providing MCCRES results to 
the unit evaluated. 

Eighth Question: (8) Who should be made aware of MCCRES 

results? 

The question was evaluated based on selection of one of 
three responses: chain of command, other like units, or all 
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Results of Question Six 



TABLE 6.7 



Results of Question Seven 



High 

# of Weighted 

Responaents/Score 

0 ( 0 ) 



INTENSITY 

Medium 

# of Weighted 

Respondents/Score 

0 ( 0 ) 



Low 

n of Weighted 

Respondents/ Sc ore 

0 ( 0 ) 



( 0 ) 



( 6 ) 



18 



(54) 15 



( 2 ) 



( 0 ) 



( 0 ) 



( 0 ) 



(30) 1 



( 0 ) 



0 



( 0 ) 0 ( 0 ) 0 



( 0 ) 



(60) 



(32) 



( 0 ) 



PERCENTAGES 



0 



2.1 



6 4 



91.5 



0 



(93) 100.0 



37 



Response Rate: 100% 



units. As indicated in Table 6.8, 60.2% of the respondents 
indicated that all units should be made aware of MCCRES 
results . 

The responses to the questions are summarized for the 
reader in Tables 6.9 and 6.10. Table 6.9 shows the distribu- 
tion by rank of the specific responses to the eight questions. 
Table 6.10 shows the distribution by community (ground/air) . 

A more detailed display of data by occupational specialty 
was available to the researchers; however, to protect the 
anonymity of the respondents only the general categorizations 
of answers is displayed. In reviewing the data in Tables 6.9 
and 6.10 there is no direct evidence of bias by any specific 
rank or community. This same result was true when evaluating 
the data by occupational specialty. 

B. ANALYSIS OF GENERAL RESPONSES TO QUESTIONS 

The primary purpose of this research. was to gain insight 
and new ideas. To this end the respondents were specifically 
asked to expand upon and explain their answers. Their 
explanations resulted in the following information. The 10% 
decision rule discussed above was used to determine which 
information to present. 

In response to Question One, which related to the uses 
of the computerized results, the majority of the inter- 
viewees said that the results from a MCCRES found their way 
into unit training plans and schedules. One of the most 
serious concerns brought out in response to this question con- 
cerned timing [Ref. 22] of the evaluation. If the MCCRES is 
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TABLE 6.8 



Results of Question Eight 



NSE 



t of 
nand" 



High 

# of Weighted 

Respondents/Score 

5 (15) 



INTENSITY 

Medium 

# of Weighted 

Respondents/Score 

6 ( 12 ) 



Low 

# of Weighted 

Respondents/Score 

0 ( 0 ) 



r like 



(2) 3 (6) 0 



( 0 ) 



TltS” 



11 



(33) 10 (20) 0 



( 0 ) 



VL: 



(50) 



(38) 



( 0 ) 



3er Responding. 36 



Response Rate; 97.3°/» 



PERCENTAGES 



30.7 



9.1 



60.2 



( 88 ) 1000 



TABLE 6.9 



Distribution by Rank of Responses to Questions 



Question # 


Colonel 


Lt. Colonel 


Major 


Captain 


TOTAL 


1. a. 


None 


0 


4 


1 


0 


5 


b. 


Planning 


4 


11 


4 


3 


22 


c. 


Historical 


0 


3 


4 


0 


7 


2. a. 


Positive 


3 


16 


7 


3 


29 


b. 


Negative 


0 


4 


2 


0 


6 


3. a. 


Positive 


1 


7 


2 


1 


11 


b. 


Negative 


2 


11 


6 


0 


19 


4. a. 


Positive 


3 


8 


3 


1 


15 


b. 


Negative 


0 


9 


3 


0 


12 


5. a. 


Positive 


2 


12 


6 


0 


20 


b. 


Negative 


0 


5 


1 


1 


7 


6. a. 


Positive 


3 


10 


7 


1 


21 


b. 


Negative 


1 


9 


4 


1 


15 


7. a. 


Oral 


0 


0 


0 


0 


0 


b. 


Written 


0 


1 


0 


0 


1 


c. 


Progressive 


0 


2 


0 


0 


2 


d. 


Combination 


4 


17 


10 


3 


34 


e. 


Other 


0 


0 


0 


0 


0 


8. a. 


Chain 


1 


7 


2 


1 


11 


b. 


Like 


0 


1 


2 


1 


4 


c. 


All 


3 


10 


7 


1 


21 
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TABLE 6.10 



Distribution by 


Community 


of Responses to Questions 


Question # 


Ground 


Aviation 


TOTAL 


1. a. None 


4 


1 


5 


b. Planning 


15 


7 


22 


c. Historical 


2 


5 


7 


2. a. Positive 


21 


8 


29 


b. Negative 


1 


5 


6 


3. a. Positive 


7 


4 


11 


b. Negative 


11 


3 


19 


4. a. Positive 


9 


6 


15 


b. Negative 


6 


6 


12 


5. a. Positive 


12 


8 


20 


b. Negative 


3 


4 


7 


6. a. Positive 


12 


9 


21 


b. Negative 


9 


6 


15 


7. a. Oral 


0 


0 


0 


b. Written 


1 


0 


1 


c. Progressive 


2 


0 


2 


d. Combination 


19 


15 


34 


e. Other 


0 


0 


0 


8. a. Chain 


9 


2 


11 


b. Like 


2 


2 


4 


c. All 


11 


10 


21 



KEY: Ground: Infantry, Tanks, Artillery, Engineer 
Aviation: Fixed wing, Helicopter, Air Control 
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given sixty days or more prior to deployment, then the unit 
may experience substantial turnover of key personnel and 
nullify the results of the MCCRES by changing key players. 

The alternative to this presents a serious dilemma. If the 
MCCRES is given too close to time of deployment, then little 
time exists to correct all but the most glaring deficiencies. 
Additionally, since no "real" second evaluation is made of 
the shortcomings, the USMC may be deploying units who have 
not in fact corrected shortcomings identified during MCCRES. 

A second major problem identified by the respondents is 
the timeliness with which the unit gets the actual MCCRES 
report. Some organizations present at least a rough of the 
printout to the unit at the time of the formal debrief, usually 
12-24 hours after termination of the MCCRES. Other units do 
not receive their formal printout until there is a complete 
staffing by division/wing, regiment/group; and in at least two 
cases the unit did not receive their formal feedback until 
they were on their deployment. One unit did not see their 
printout at all after the MCCRES. (The last comment is pro- 
vided as an exception to the 10% rule because of the serious- 
ness of this situation.) 

Responses to Question Two indicated that the current 
MCCRES gives sufficient information to improve combat readi- 
ness. In the case of the rotory wing MCCRES, three of the six 
rotory wing respondents indicated that more weight was needed 
on flying skills than is currently given by the MPS in Volume 
III. Generally the responses from the rotory wing community 
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indicated too much emphasis is given to non-flight require- 
ments and too little to flight skills. In addition responses 
from the fixed-wing community indicated that technology is 
changing more rapidly than the MCCRES. With the implementa- 
tion of the FA-18 aircraft and changes to other aircraft, the 
MCCRES evaluation needs to be changed to be current and realis 
tic. In other words the aircraft can do a lot more than 
MCCRES requires. Discussions with the Readiness Branch indi- 
cate they are well aware of the problems caused by changing 
technology in the FA-18 and other new aircraft, and are 
presently improving the MCCRES requirements to test the air- 
craft at limits that more closely approximate its actual 
capabilities rather than against the less demanding F-4 
Requirements . 

A total of 13 respondents commented that MCCRES results 
are only as credible as the evaluators. The absence of 
standardization of evaluators and evaluator training, as well 
as the variances in how organizations pick evaluators had a 
great impact on the value perceptions and the worth of the 
MCCRES as a measure of the unit's readiness. This was ex- 
pressed by six of the fifteen officers of the aviation com- 
munity. Forty percent of the respondents submitted an 
unsolicited comment on the quality of the evaluators. If 
the evaluator is not regarded as highly qualified by his peers 
then the evaluation is regarded with little weight. The 
question of evaluator selection and bias is discussed in 
more depth by Conatser [Ref. 6] and Wheeler [Ref. 5]. 
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A final problem cited dealt with the MCCRES concept of com- 
paring unit performance with published combat doctrine. Doc- 
trine as codified in written manuals may be outdated. If so, 
the USMC is evaluating units by comparing their performance to 
standards which would not be employed on the modern battlefield 
An example of this cited by four infantry commanders is "maneuv 
warfare." Maneuver warfare is widely accepted by Western mili- 
tary analysts as the most promising way to fight and win any 
future war against Soviet tactics. However, maneuver warfare 
is not written into current Marine Corps doctrinal publications 
This shortfall points out the need for distribution of MCCRES 
results to all training and doctrine commands to ensure they 
are aware of the strengths and shortfalls that units/individual 
display on MCCRES. 

The most common response to Question Three, which related 
to effectiveness of the display of the results, was negative. 

A percentage of 64.6% of the respondents said that they did 
not feel the computerized printout is the best way to dis- 
play MCCRES results. Nineteen of thirty respondents indicated 
more subjective data was needed to fully understand the 
units' evaluations and to be able to take corrective actions. 
The computer printout provides a "Yes/No" evaluation, but it 
fails to provide the "why" and "what" [Ref. 37 J . More details 
are needed to describe specific reasons for "No" marks and 
recommendations are necessary to help the unit find ways to 
improve. All those responding negatively (19 of the 30 
respondents) indicated it was much more valuable to spell out 
the details of a shortfall and to suggest corrective action, 
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than to simply provide a "Yes/No" on an evaluation sheet. One 
wing MCCRES officer accomplishes this by writing all comments 
directly on the MCCRES printout before it is provided to the 
unit. The wing MCCRES officer said this facilitates a much 
more complete understanding of problem areas. The general 
consensus of the 30 respondents to this question was that the 
subjective, explanatory information is by far the most valua- 
ble to the evaluated unit and the percentile scores on the 
printout are probably of greater value to higher level head- 
quarters for use in planning future training. 

In response to Question Four, which related to the under- 
standability of the results, 15 of the 27 respondents or 
55.6% agreed that the current format is not difficult to 
understand. In general those who indicated a willingness to 
reference the directives and compare the printout to the appro- 
priate Volume were satisfied with the explanations provided 
therein. Those who said the current format is difficult to 
understand (12 of the 27 or 44.4%) did so based on a criterion 
of the inconvenience of the system, that is requiring both 
the printout and the Volume. The combined use of the printout 
and Volume requires a good bit of tedious research to find the 
details of each MPS/Task/Requirement . The recent addition of 
a description of each MPS/Task/Requirement on the MCCRES 
printout should help reduce this problem. On the other hand, 
the brief description may cause even less referencing of the 
source document to obtain complete details. One suggestion 
was that key players from the unit to be evaluated should be 
given detailed classes and instructions on the MPS several 
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weeks prior to a scheduled MCCRES. This has recently been 
started in one wing and the results are excellent. This 
contention was supported by both squadron commanders ques- 
tioned as well as by those in the Readiness Branch at Head- 
quarters Marine Corps. Just as there is a need to educate 
evaluators, there is also a need to educate those to be 
evaluated [Refs. 5,27,28]. 

An added point made by five unit commanders was that the 
results must get out to all those concerned. If the response 
to an evaluation is delayed or the results simply become his- 
torical record, those who need to know at company, platoon, 
and squad level are never made aware of the results. Emphasis 
is needed on informing all participants down to the individual 
Marine of the results of MCCRES [Refs. 16,39]. MCCRES re- 
sults should be distributed to all detachments to ensure 
they know how their performance was evaluated. 

Responses to Question Five indicated that the orders 
generally seem well written and easy to interpret. The same 
suggestion for education made on Question Four applies here. 
That is, any instruction given to key unit players will result 
in better overall understanding of MCCRES and allows the unit 
to better use the results. 

Several specific shortcomings in MCCRES were also noted 
in response to Question Five. Those relating to aviation were 
addressed under Question Two. One infantry division has 
found some serious shortcomings, specifically in the failure 
to provide for intelligence gathering, fire support coordinatic 
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and allocation of tasks from higher" headquarters . These three 
areas all apply to specific MPS ' s that need expansion or 
improvement . 

Question Six, which related to unit results comparisons, 
by far evoked the most emotional and intense responses. Any 
time a comparative index is proposed it has the potential for 
both good and bad results depending on its use. Of 37 indi- 
viduals interviewed, 55.4% favored some sort of use of com- 
parative scores. Those opposed tended to do so based on the 
view that any comparison of scores provided a potential for use 
as a report card to compare units and specifically unit com- 
manders. The primary objections to the use of comparative 
scores was based on the lack of overall standardization within 
MCCRES , especially the differences in evaluators, scenarios, 
terrain, and weather conditions. 

Those who favored comparative scoring did so for various 
reasons. Four respondents thought comparative scoring had value, 
but only for higher headquarters, training, and doctrine commands. 
Seven others thought comparative scoring had value if shown to 
the unit in complete privacy. A final segment of ten respondents 
indicated that the comparative scoring had potential to create 
more unit competition. Thus, overall, 10 out of the 37 respon- 
dents supported any sort of mass publication of comparative scores. 

The results to Question Seven provided evidence that 
feedback must be provided in various forms throughout the 
exercise to be most effective. As seen in response to Question 
Three, 64.6% wanted more subjective feedback. In Question 
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Seven that was reaffirmed. The respondents said that by far th 
most valuable feedback was obtained from verbal, subjective 
critiques given immediately after an event has taken place. 

A total of 91.5% of the respondents indicated a combination 
of debriefs and written results is most desirable. The written 
results should not only include the objective numerical re- 
sults of "Yes/No" markings, but also explanation of all the 
"No/N.A." markings. 

Question Eight, which dealt with the distribution of MCCRES 
results, was most often answered with "all units," 60.2% of 
the time. The specific point made here was that the chain of 
command has an obvious need to know the results. All units 
could benefit from a generic trend analysis report. Trend 
analysis would provide a cross pollination of critical infor- 
mation throughout the Marine Corps and assist those involved 
in developing training and doctrine. Those favoring "all 
units" or "other like units" comprised a total percentage of 
69.3%, provided a favorable response to the suggestion of re- 
ceiving a generic trend analysis at regular intervals from 
the MCCRES Data Bank. The fact that no respondent provided any 
objection to this idea is indicative of its positive value. 

C. SUMMARY 

This chapter presented the results of the field study. 

As can be seen in the next chapter the analysis of specific 
and general responses to the eight questions provided answers 
to the research questions posed by this study. In addition 
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a number of important unsolicited comments, outside the scope 
of this study, were also obtained. These comments are pre- 
sented in Appendix B. 



10 3 



VII. DISCUSSION AND CONCLUSIONS 



A. ANSWERS TO PRIMARY RESEARCH QUESTIONS 

The results of the research have provided answers to 
questions posed at the beginning of this project. The three 
primary research questions were: 

(1) After MCCRES, what information should be fed back 
to the evaluated unit commander? 

(2) In what form should the evaluation feedback be? 

(3) What channel (s) should be used to provide the 
feedback to the evaluated unit? 

A comprehensive feedback model is presented which incorporates 
the results for the three questions. 

It was evident throughout this study that the unit com- 
mander faces many constraints: training time, material, 

equipment, logistics, and personnel. The information from 
MCCRES should be provided in such a way as to allow for 
better allocation of these scarce resources to the unit. 

As discussed in Chapter II, the overall goal of the MCCRES 
is not only to provide a measure of unit readiness, but to 
provide effective feedback to the organization to enhance the 
opportunities to improve readiness by applying resources to 
areas identified as deficient by MCCRES. The discussion of 
management control and evaluation, as presented in Chapter 
III, provides a rich background for understanding how to 
perform the MCCRES evaluation and how to effectively use the 
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results of an evaluation. In models of evaluation, informa- 
tion and feedback play an important role in both performing 
and using the results of an evaluation. In general the model 
of evaluation have common desirable characteristics such as: 
brevity, clarity, timeliness, interim products and reports, 
and responsiveness of feedback [Ref. 35J . 

Additionally, characteristics of the organizational con- 
trol system [Ref. 44] such as goals, standards, measurement 
system, evaluation and reward system as well as the feedback 
loop play an equally important role in the evaluation models 
[Refs. 32,34,35,33, and 40]. 

Based upon these findings and the results of a field 
study, a model for MCCRES feedback was developed. The model 
is presented in Figure 7.1. Each of the eight major elements 
of the model that affect the evaluated unit are explained 
below . 



Pre-MCCRES Brief 




Figure 7.1 Model of MCCRES Feedback to an Evaluated Unit. 



105 



1 . Pre-MCCRES Briefing 



A detailed orientation several weeks prior to MCCRES 
to provide a firm foundation for all key personnel. The 
orientation should provide an explanation of appropriate 
Volumes of MCO 3501.2. Explanation of the details of each 
MPS and procedures for the evaluator allows the unit to 
maximize the potential from the MCCRES feedback [Ref. 37] . 

2 . Real Time Feedback 

Regular verbal feedback throughout the exercise by 
evaluators at each level. At any logical break in the 
scenario detailed evaluations of the most recent events 
should be provided by the evaluators. This provides data 
while the event is fresh in everyone's minds [Ref. 35]. 

3 . Evaluator Worksheets 

The Evaluator Worksheets should be provided directly 
to the evaluated element shortly after conclusion of the 
exercise. This provides an interim evaluation report and 
allows for maximum feedback to the specific element evaluated. 
It would be especially valuable to subunits and attachments 
who often receive only minimal feedback under current proce- 
dures [Ref. 37] . 

4 . Two-way Debrief 

At the end of exercise, twelve to twenty-four hours 
after the end of the formal MCCRES, a detailed debrief should 
be held for evaluators and key players from the unit evaluated. 
Opportunities should be given for two-way discussion to resolve 
any confusion or disagreement particularly regarding subjective 
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areas. The individuals from the evaluated unit should leave 
with not only a clear understanding of the deficiencies, 
but also with a clear understanding of what caused the "No" 
evaluations and suggestions for improvement [Ref. 35] . Since 
the commander of the unit evaluated is responsible for the 
employment of the feedback, he should have the flexibility 
to tailor the debrief to his leadership style. The commander 
may desire to have only a few selected evaluators and officers 
present or the unit commander may desire all evaluators and 
unit officers to be present. 

5 . Computerized MCCRES Printout 

The computerized MCCRES printout with percentile 
scores and detailed amplifying remarks should be provided to 
the unit in a timely manner [Ref. 35] . It does a unit little 
good to get the detailed results when there is no time to 
correct deficiencies. What is timely can vary depending on 
deployment schedules, but getting a copy of the results in 
the hands of the evaluated unit so that action can be taken 
is essential. It is inexcusable and unprofessional to allow 
staffing to delay this critical information. Delays also 
put the unit further away from the MCCRES and it tends to 
become an historical document vice a map for future training. 

6 . Comparative Percentile Results 

The data needed to compare units within a division or 
wing is currently available with limited research within the 
division or wing files. Additionally most division or wing 
headquarters are well aware of how their scores compare with 
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other like units. Usage of the general MCCRES data bank to 
provide comparative scores, medians, and standard deviations 
should be provided to major headquarters staff, training, and 
doctrine commands for the limited purpose of planning future 
resource allocation and to develop training doctrine. These 
comparative results should be limited to Section/MPS/Task 
scores and should not include any sort of comparative final 
score for the units as a whole. Providing any sort of overall 
scores that could be used to compare individual units tested 
could well have a negative, impact because of the many variation 
in evaluation conditions that exist. It is further recommended 
that these comparative results be directed only at division/ 
wing level or higher commands to reduce the chance of any 
type of unit comparisons. 

An example of comparative percentile results that could 
be provided to division/wing level for use in planning future 
training is presented in Figure 7.2. 



PAST SIX MONTH MCCRES RESULTS 









Division/ 

Wing 


Overall 
Marine Corps 


Section 


2.0 


Operations Performed 


93.2% 


91.1% 


MPS 


2.A 


Actions by Marines 


97.3% 


98.4% 


Task 


2.A.1 


Discipline 


86.4% 


90.2% 



Figure 7.2. Sample of Comparative Summary Percentile Results 
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7 . Follow-up Report 



Currently no follow up evaluation is made to confirm 
that the deficiencies have been corrected. Some type of 
final check is needed. This check would best be accomplished 
with little formality. Since the unit commander has the 
responsibility for application of MCCRES feedback to improve 
unit readiness, it is best left to the unit commander to do 
the final check which certifies that deficiencies have been 
corrected. Resource reallocation from outside the evaluated 
unit likewise should be certified at the appropriate level. 

The key issue here is that the unit must be deployed at the 
highest level of combat readiness and verification of correc- 
tion of MCCRES deficiencies at least on an informal basis is 
essential. This follow-up should be made as simply as possi- 
ble. The unit commander can appropriately perform this 
certification based on his own experience and judgment. A 
formal follow-up inspection is neither necessary nor desirable. 

8 . Trend Analysis 

To gain a higher level of across-the-board readiness, 
a trend analysis would be of value to all units. The MCCRES 
data base represents a resource that could be used to keep 
the Marine Corps informed of its overall strengths and 
weaknesses. Special emphasis should be made on getting these 
results to training and doctrine organizations throughout 
the Marine Corps. 
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B. ANSWERS TO SECONDARY RESEARCH QUESTIONS 

The results of the research provided answers to the 
secondary research questions posed at the beginning of the 
project. The three secondary research questions were: 

(1) What is the validity of comparing MCCRES results 
given under different conditions? 

(2) Does MCCRES contribute to effective training? 

(3) Is there available time and follow up to ensure 
correction of deficiencies identified during MCCRES? 

A first response to the question of the validity of com- 
paring MCCRES results given under differing conditions might 
be to say that such comparisons can only be made if complete 
standardization is achieved. Although complete standardiza- 
tion for purposes of evaluation may seem to be desirable, this 
may not be feasible. Since various Marine Corps units (east 
coast, west coast, overseas) are preparing for different mis- 
sions, it is desirable to simulate their expected combat 
employment through the use of different scenarios, terrain, 
and even MPS's. Additionally the Marine Corps has other exer- 
cises, such as the Combined Arms Exercise (CAX's) , which are 
used to evaluate under standard scenario, terrain, and evalua- 
tors, but they have a more limited purpose than MCCRES. 
Additionally, because of the broader purposes of MCCRES, 
complete standardization is probably not desirable. For its 
intended purpose, MCCRES appears to provide a relatively high 
degree of objectivity when compared with other military systems, 
such as the performance evaluation system used for the 
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promotions of Marine Officers, and the results are likely to 
be useful, at least for generic comparisons, and to identify 
trends . 

In response to the question of MCCRES ' s contribution to 
effective training, it is evident from the data gathered for 
this study that MCCRES has the potential to, and currently 
does, improve unit training. Many units train to MCCRES 
standards and go so far as to use actual MCCRES Section/MPS/ 
Task/Requirements to structure their unit training. There- 
fore as the Individual Training Standards (ITS) are developed 
for all units, it is essential that they be closely compared 
with MCCRES standards so that the training and evaluation 
development move in the same direction [Refs. 5,30,31], 

The third question, time availability to correct MCCRES 
deficiencies, has previously been discussed. No doubt more 
time would allow for more training in deficient areas. 
However, the question of follow up is also critical. As 
stated above, at least an informal re-validation of deficient 
areas is needed to ensure the effectiveness of the MCCRES. 

C. SUMMARY 

This research has provided detailed answers to the ques- 
tions posed at the beginning of the study. The feedback to 
the unit is the key element to making the MCCRES of value to 
the unit evaluated. The purpose of MCCRES is to provide 
timely and accurate evaluation of the force readiness. Key 
to improving readiness is identification of strengths and 
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weaknesses, and providing timely and thorough feedback to the 
evaluated unit and its chain of command. The more timely 
and thorough the feedback, the higher the likelihood of im- 
proved future readiness. The MCCRES is currently viewed as 
a sound, valuable evaluation system. However, as this study 
has demonstrated, the feedback to the units can be improved. 



D. STUDY RECOMMENDATIONS 

The feedback model developed and displayed as Figure 7.1 
should be incorporated as the minimum acceptable procedure 
for providing feedback from any MCCRES. To accomplish this 
the eight steps in the feedback model should be followed 
during each evaluation. The following information should 
be routinely provided as feedback. 

Pre-MCCRES Briefing — to key MCCRES players from the 
evaluated unit 

Real Time Feedback — distributed to all those evaluated 

Two-way Debrief at end of exercise--tailored to needs of 
the unit commander to ensure full understanding of 
evaluation results 

Computerized MCCRES Printout — provided in a timely manner 
to allow new resource allocation prior to deployment 

Comparative Percentile Results — to help direct unit train- 
ing and for incorporation by individual training and 
doctrine commands 

Follow-up Report — done informally by unit commander to 
ensure closure of feedback loop by correction of 
identified deficiencies 

Trend Analysis — to support future training as well as 
changes in doctrine. 

Implementation of these recommendations would suit the 
stated needs of fleet units as well as serve the intended 
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MCCRES purpose of providing an evaluation of unit readiness 
to higher headquarters. 

E. SUMMARY OF STUDY 

The purpose of this study was to determine the most 
appropriate and effective manner in which to provide feedback 
to the unit commander from a MCCRES. To this end research was 
conducted into the areas of management control and evaluation 
theory. Additionally , specif ic research was conducted in 
the area of existing systems which measure or evaluate 
readiness . 

The study was approached from a multi-disciplinary back- 
ground with a detailed emphasis on the economic question of 
how better distribution could be made of the scarce resources 
of manpower, equipment, and training time based on the re- 
sults of MCCRES. To answer the questions a detailed field 
study was conducted in which interviews were completed with 
37 Marine officers of rank from Captain to Colonel who have 
had involvement with MCCRES as key billet holders. Addi- 
tionally much of the original documentation that laid the 
groundwork for the development of MCCRES was reviewed along 
with interviews and visits with some of the original developers 
of the system. 

The results of the study are displayed as an eight step 
feedback model which is based on previous research in the 
fields of management control and evaluation, as well as the 
results of the field study. These results are communicated 
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to the reader as a model in Figure 7.1 which is comprised of 
the following elements: 

(1) Pre-MCCRES Briefing 

(2) Real Time Feedback 

(3) Evaluator worksheets 

(4) Two-way Debrief at the end of Exercise 

(5) Computerized MCCRES Printout 

(6) Comparative Percentile Results 

(7) Follow-up Report 

(8) Trend Analysis 

The incorporation of this model as standard MCCRES 
feedback procedure will significantly enhance the value 
of the results to the evaluated unit and will improve the 
understanding of the resource allocation needs at all 
1 eve 1 s . 
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APPENDIX A 



SAMPLE MARINE CORPS COMBAT READINESS EVALUATION 
SYSTEM SOFTWARE PRINTOUT 



2.0 




INFANTRY 




BY SECTION 


SECTION 


WEIGHT 


EVAL 


CUM WT 


A) 


OPNPERFORM 


100.00 


95 


68.97 


B) 


STDTACTICS 


45.00 


92 


31.03 


C) 


SPECIALOPS 


18 .00 


N/A 


0.00 


D) 


EXTERSUPTD 


18.00 


N/A 


0.00 




VOLUME TOTAL 


100.00 


94 


100.00 


2.0 




INFANTRY 




BY MPS 


2. A 




OPNPERFORM 






MPS 




WEIGHT 


EVAL 


CUM WT 


1) 


ACTNSBYMAR 


100.00 


92 


33.16 


2) 


CMD-CONTRL 


58.00 


95 


19.23 


3) 


FSUPTCOORD 


50.00 


100 


16.58 




SECTION TOTAL 


100.00 


95 


68.97 


2 .B 




STDTACTICS 






MPS 




WEIGHT 


EVAL 


CUM WT 


1) 


SURFASSLT 


100 .00 


N/A 


0.00 


2) 


HELIASSLT 


80 .00 


N/A 


0.00 


3) 


MVTOCONTCT 


32.00 


100 


4.97 


4) 


ATTACK 


80.00 


96 


12.41 


5) 


NIGHTATTCK 


28.00 


68 


4 . 34 


6) 


DEFENSE 


60.00 


93 


9 . 31 


7) 


RETROGRADE 


20 .00 


N/A 


0.00 


8) 


TACP 


75.00 


N/A 


0.00 




SECTION TOTAL 


45.00 


92 


31.03 
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2.C 




SPECIALOPS 


N/A 




2 . D 




EXTERSUPTD 


N/A 




2.0 




INFANTRY 




BY TASK 


< 

• 

(N 




OPNPERFORM 






2. A. 


.1 


ACTNS BY MAR 






TASK 


WEIGHT 


EVAL 


CUM WT 


1) 


DISCIPLINE 


100.00 


100 


3.64 


2) 


DISPERSION 


91.00 


100 


3.31 


3) 


COVER 


100.00 


100 


3.64 


4) 


CAMOUFLAGE 


82.00 


100 


2.98 


5) 


SECURITY 


100.00 


100 


3.64 


6) 


RECONPATROL 


64.00 


84 


2.33 


7) 


COMBATPTRL 


55.00 


100 


2.00 


8) 


ELECTRONIC 


82.00 


43 


2.98 


9) 


NBC 


55.00 


N/A 


0.00 


10) 


CHEMICAL 


55.00 


100 


2.00 


ID 


ENSEMYAIR 


82.00 


100 


2.98 


12) 


POWS 


55.00 


80 


2.00 


13) 


CASUALTIES 


45.00 


80 


1.64 




MPS TOTAL 


100.00 


92 


33.16 
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2.0 


INFANTRY 




BY REQUIREMENT 


2 .A 


OPNPERFORM 






2.A.1 


ACTNSBYMAR 






2. A. 1.1 


DISCIPLINE 






REQUIREMENT 


WEIGHT 


EVAL 


CUM WT 


1) SELF 


100.00 


10 0 


0 . 53 


2) WPNS 


80 .00 


100 


0 .42 


3) MAINT 


80 .00 


100 


0.42 


4) FIRE 


100.00 


100 


0.53 


5) SUPPLY 


50.00 


N/A 


0.00 


6) COMM 


100.00 


100 


0.53 


7) NOISE 


90.00 


10 0 


0.47 


8) LIGHT 


90.00 


100 


0.47 


9) HYGENIC 


50.00 


100 


0.26 


TASK TOTAL 


100.00 


100 


3.64 


2. A. 1.2 


DISPERSION 






REQUIREMENT 


WEIGHT 


EVAL 


CUM WT 


1) INDIVIDUAL 


100.00 


10 0 


0 . 82 


2) UNIT 


80.00 


100 


0.66 


3) LEADERS 


100 .00 


10 0 


0 . 82 


4) MATERIEL 


48.00 


100 


0 .39 


5) WEAPONS 


76.00 


100 


0.62 


TASK TOTAL 


91.00 


100 


3.31 
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APPENDIX B 



SUMMARY OF UNSOLICITED COMMENTS 

Because of the technique employed of encouraging expan- 
sion and discussion on each question, numerous ideas were 
brought forth. Although these were not specifically evaluated, 
they are of potential importance and should be at least con- 
sidered by anyone interested in MCCRES . The comments identi- 
fied herein were selected based on their logic and potential 
impact on MCCRES. No attempt was made to support or disprove 
these remarks; such an analysis is, therefore, left for 
future research. 

A. PRESENT APPROACH TO MCCRES 

Few if any units fail. MCCRES credibility is in serious 
doubt to several respondents because all units seem to pass 
with very similar overall scores. As a result of this, the 
final MCCRES grade is not very meaningful, especially without 
other grades with which to compare the score. As a result 
of this, two of those questioned indicated they saw no value 
at all to overall unit grade. They stated consideration should 
be given to doing away with the unit grade and just providing 
comparative-trend analysis scores on a generic basis. This 
would do away with the informal comparison of scores that now 
takes place and provide better support of the intended purpose 
of not using the MCCRES to compare the overall readiness of 
units evaluated. 
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B. TWO MCCRES'S 



The general feeling among all the respondents is that 
MCCRES is a superb opportunity for training with more assets 
allocated to the evaluated unit than for most other exercises. 
Some suggest a MCCRES should be conducted at both the be- 
ginning and end of a training cycle; to evaluate improvements 
and to measure the unit against itself. This would allow 
evaluators to be more critical, because the unit would iden- 
tify strengths and weaknesses early and adjust training 
objectives accordingly. Although it is a costly suggestion, 
it has potential to further improve readiness. 

C. ANALYZE INDIVIDUALS 

The Command needs to know not only strengths and weak- 
nesses of unit/subunit performance, but the evaluated unit 
commander also needs an independent assessment of individual 
leadership strengths and weaknesses. Providing MPS ' s that 
require individual evaluation of leaders at all levels would 
assist the unit commanders to reinforce his own observations 
and to be more aware of where he needs to place emphasis. 

D. TACTICAL STERILITY 

MCCRES rewards those who follow doctrine explicitly and 
avoid innovation. In maneuver warfare innovation and flexi- 
bility are the key to success. Yet MCCRES measures perform- 
ance by comparison with standard doctrine. Problems here 
arise not out of the MCCRES procedure, but the fact that much 
of the doctrine upon which MCCRES is based rapidly is being 
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outdated. To have an effective/believable evaluation, the 
whole cycle of doctrine and inclusion of new doctrines in 
MCCRES must be maintained. The general thought is MCCRES 
rewards those who "play the game" and may punish those who 
are "ahead of written doctrine." Some respondents said 
sufficient strides have been made to overcome this in certain 
situations, but others said this remains a widespread system 
problem. 

E. LEVEL OF APPLICATION 

Although Reference 1 states that the Fleet Marine Force 
Commanding General is responsible for application of MCCRES, 
the authority is delegated to various levels. Some of the 
control is held tightly at the division/wing levels; others 
delegate the actual evaluation to regiment/group level. 

These differing procedures greatly change the standardization 
of evaluation and reduce the pool of potential evaluators. 

F. GROUPING AND TIMING FOR EVALUATION 

Most every MCCRES has a different grouping of unit types. 
For instance some organizations evaluate battalion- landing 
team units with the attached artillery, engineers, tanks. 

Others evaluate only the infantry battalion; while some organi- 
zations use the MCCRES to evaluate artillery as battalions 
and others do not. The Marine Corps standard is that each 
unit be evaluated using MCCRES every two years, to ensure 
that all units in fact are evaluated. A detailed scheduling 
system should be incorporated to ensure that no units are 
missed in the process. 
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